Method and system for controlling space magnification for stereoscopic images

ABSTRACT

The invention relates to a method and system of displaying stereoscopic images. The method comprises displaying at least one stereoscopic image on a set of display device, the stereoscopic image comprising a pair of two-dimensional plane images, and providing adjustment data for spatial magnification, the spatial magnification relating to a size of a space that is imaged. The method also comprises transmitting the magnification adjustment data to a set of remotely located stereoscopic cameras, and adjusting the distance between the stereoscopic cameras based on the magnification adjustment data.

RELATED APPLICATIONS

[0001] This application is a continuation application, and claims thebenefit under 35 U.S.C. §§120 and 365 of PCT application No.PCT/KR01/01398 filed on Aug. 17, 2001 and published on Feb. 21, 2002, inEnglish, which is hereby incorporated by reference herein. Thisapplication is related to, and hereby incorporates by reference, thefollowing patent applications:

[0002] U.S. patent application entitled “METHOD AND SYSTEM FORCALCULATING A PHOTOGRAPHING RATIO OF A CAMERA”, filed on even dateherewith and having application Ser. No. ______(Attorney Docket No.GRANP2.001C1);

[0003] U.S. patent application entitled “METHOD AND SYSTEM FORCONTROLLING A SCREEN RATIO BASED ON A PHOTOGRAPHING RATIO”, filed oneven date herewith and having application Ser. No. ______(AttorneyDocket No. GRANP2.001C2);

[0004] U.S. patent application entitled “METHOD AND SYSTEM FORCONTROLLING THE DISPLAY LOCATION OF STEREOSCOPIC IMAGES”, filed on evendate herewith and having application Ser. No. ______(Attorney Docket No.GRANP2.001C3);

[0005] U.S. patent application entitled “METHOD AND SYSTEM FOR PROVIDINGTHE MOTION INFORMATION OF STEREOSCOPIC CAMERAS”, filed on even dateherewith and having application Ser. No. ______(Attorney Docket No.GRANP2.001C4);

[0006] U.S. patent application entitled “METHOD AND SYSTEM FORCONTROLLING THE MOTION OF STEREOSCOPIC CAMERAS BASED ON A VIEWER'S EYEMOTION”, filed on even date herewith and having application Ser. No.______(Attorney Docket No. GRANP2.001C5);

[0007] U.S. patent application entitled “METHOD AND SYSTEM OFSTEREOSCOPIC IMAGE DISPLAY FOR GUIDING A VIEWER'S EYE MOTION USING ATHREE-DIMENSIONAL MOUSE”, filed on even date herewith and havingapplication Ser. No. ______(Attorney Docket No. GRANP2.001C6);

[0008] U.S. patent application entitled “METHOD AND SYSTEM FORCONTROLLING THE MOTION OF STEREOSCOPIC CAMERAS USING A THREE-DIMENSIONALMOUSE”, filed on even date herewith and having application Ser No.______(Attorney Docket No. GRANP2.001C7);

[0009] U.S. patent application entitled “METHOD AND SYSTEM FOR ADJUSTINGDISPLAY ANGLES OF A STEREOSCOPIC IMAGE BASED ON A CAMERA LOCATION ”,filed on even date herewith and having application Ser. No._____(Attorney Docket No. GRANP2.00C9);

[0010] U.S. patent application entitled “METHOD AND SYSTEM FORTRANSMITTING OR STORING STEREOSCOPIC IMAGES AND PHOTOGRAPHING RATIOS FORTHE IMAGES”, filed on even date herewith and having application Ser. No.______(Attorney Docket No. GRANP2.001C10); AND

[0011] U.S. patent application entitled “PORTABLE COMMUNICATION DEVICEFOR STEREOSCOPIC IMAGE DISPLAY AND TRANSMISSION”, filed on even dateherewith and having application Ser. No. ______(Attorney Docket No.GRANP2.00C11)

BACKGROUND OF THE INVENTION

[0012] 1. Field of the Invention

[0013] The present invention relates to a method and system forgenerating and/or displaying a more realistic stereoscopic image.Specifically, the present invention relates to a method and system foradjusting a size of a space, which a viewer perceives, that is imaged bya set of stereoscopic cameras, by adjusting the distance between a setof stereoscopic cameras.

[0014] 2. Description of the Related Technology

[0015] In general, a human being can recognize an object by sensing theenvironment through eyes. Also, as the two eyes are spaced apart apredetermined distance from each other, the object perceived by the twoeyes is initially sensed as two images, each image being formed by oneof the left or right eyes. The object is recognized by the human brainas the two images are partially overlapped. Here, in the portion wherethe images perceived by a human being overlap, as the two differentimages transmitted from the left and right eyes are synthesized in thebrain, there is a perception of 3-dimensions.

[0016] By using the above principle, various conventional 3-D imagegenerating and reproducing systems using cameras and displays have beendeveloped.

[0017] As one example of the systems, U.S. Pat. No. 4,729,017 discloses“Stereoscopic display method and apparatus therefor.” With a relativelysimple construction, the apparatus allows a viewer to view astereoscopic image via the naked eye.

[0018] As another example of the systems, U.S. Pat. No. 5,978,143discloses “Stereoscopic recording and display system.” The patentdiscloses that the stereoscopically shown image content is easilycontrollable by the observer within the scene, which is recorded by thestereo camera.

[0019] As another example of the systems, U.S. Pat. No. 6,005,607discloses “Stereoscopic computer graphics image generating apparatus andstereoscopic TV apparatus.” This apparatus stereoscopically displaystwo-dimensional images generated from three-dimensional structuralinformation.

SUMMARY OF CERTAIN INVENTIVE ASPECTS OF THE INVENTION

[0020] One aspect of the invention provides a method of displayingstereoscopic images. The method comprises displaying at least onestereoscopic image on a set of display device, the stereoscopic imagecomprising a pair of two-dimensional plane images, and providingadjustment data for spatial magnification, the spatial magnificationrelating to a size of a space that is imaged. The method also comprisestransmitting the magnification adjustment data to a set of remotelylocated stereoscopic cameras, and adjusting the distance between thestereoscopic cameras based on the magnification adjustment data.

[0021] Another aspect of the invention provides a system for displayingstereoscopic images. The system comprises a set of display device, aninput device, a transmitter, a transmitter, a receiver, a set ofstereoscopic cameras and a camera controller. The set of display devicedisplays at least one stereoscopic image, the stereoscopic imagecomprising a pair of two-dimensional plane images. The input deviceprovides adjustment data for spatial magnification, the spatialmagnification relating to a size of a space that is imaged. Thetransmitter transmits the magnification adjustment data. The receiverreceives the magnification adjustment data. The camera controlleradjusts the distance between the stereoscopic cameras based on themagnification adjustment data.

[0022] Another aspect of the invention provides a camera system forcommunicating data with a set of display device. The system comprises aset of stereoscopic cameras, a receiver and a camera controller. The setof stereoscopic cameras produce at least one stereoscopic image. Thereceiver receives adjustment data for spatial magnification from the setof display device, the space magnification relating to a size of thespace that is imaged by the set of stereoscopic cameras. The cameracontroller adjusts the distance between the set of stereoscopic camerasbased on the spatial magnification adjustment data.

[0023] Still another aspect of the invention provides a method ofcontrolling a set of stereoscopic cameras. The method comprisesproviding a set of stereoscopic cameras, and generating at least onestereoscopic image, the stereoscopic image comprising a pair oftwo-dimensional plane images generated by the set of stereoscopiccameras, respectively. The method also comprises receiving adjustmentdata for spatial magnification, the spatial magnification relating to asize of the space that is imaged by the set of stereoscopic cameras, andadjusting the distance between the stereoscopic cameras based on thespatial magnification adjustment data.

[0024] Yet another aspect of the invention provides a method ofdisplaying stereoscopic images. The method comprises providing a pair ofprojection portions spaced at a predetermined distance apart from eachother and configured to project three-dimensional structural data of ascene into a pair of two-dimensional planes. The method also comprisesproducing at least one stereoscopic image from the three-dimensionalstructural data by the pair of projection portions spaced at apredetermined distance apart from each other, the stereoscopic imagecomprising a pair of two-dimensional plane images. The method alsocomprises displaying the pair of two-dimensional plane images in a pairof display devices, respectively, and providing adjustment data forspace magnification to the display devices, the space magnificationrelating to a size of the scene. The method comprises adjusting thedistance between the projection portions based on the magnificationadjustment data.

[0025] Yet another aspect of the invention provides a system fordisplaying stereoscopic images. The system comprises a pair ofprojection portions, a pair of display devices, an input device, and acomputing device. The pair of projection portions are spaced at apredetermined distance apart from each other and produce at least onestereoscopic image from three-dimensional structural data of a scene,the stereoscopic image comprising a pair of two-dimensional planeimages. The pair of display devices display the pair of two-dimensionalplane images, respectively. The input device provides adjustment datafor spatial magnification, the spatial magnification relating to a sizeof the scene. The computing device adjusts the distance between theprojection portions based on the magnification adjustment data.

BRIEF DESCRIPTION OF THE DRAWINGS

[0026]FIG. 1A illustrates one typical 3-D image generating andreproducing apparatus.

[0027]FIG. 1B illustrates another typical 3-D image generating andreproducing apparatus.

[0028]FIGS. 2A and 2B illustrate a photographing ratio of a camera.

[0029]FIGS. 3A and 3B illustrate a screen ratio of a display device thatdisplays a photographed image.

[0030]FIG. 4A illustrates the variation of the distance between anobject lens and a film according to the variation of a focal length of acamera.

[0031]FIG. 4B illustrates the variation of a photographing ratioaccording to the variation of the focal length of the camera.

[0032]FIG. 4C shows the relationship between a photographing ratio andthe focal length of the camera.

[0033]FIG. 4D illustrates an exemplary table showing maximum and minimumphotographing ratios of a camera.

[0034]FIG. 5A illustrates a photographing ratio calculation apparatusaccording to one aspect of the invention.

[0035]FIG. 5B illustrates a photographing ratio calculation apparatusaccording to another aspect of the invention.

[0036]FIG. 6A illustrates an exemplary flowchart for explaining theoperation of the photographing ratio calculation apparatus of FIG. 5A.

[0037]FIG. 6B illustrates an exemplary flowchart for explaining theoperation of the photographing ratio calculation apparatus of FIG. 5B.

[0038]FIG. 7 illustrates a camera comprising the photographing ratiocalculation apparatus as shown in FIGS. 5A and 5B.

[0039]FIG. 8 illustrates a system for displaying stereoscopic imagessuch that a photographing ratio (A:B:C) is substantially the same as ascreen ratio (D:E:F).

[0040]FIG. 9 illustrates an exemplary flowchart for explaining theoperation of the image size adjusting portion of FIG. 8.

[0041]FIG. 10 is a conceptual drawing for explaining the image sizeadjustment in each of the display devices.

[0042]FIG. 11 illustrates an exemplary flowchart for explaining theentire operation of the system shown in FIG. 8.

[0043]FIG. 12 illustrates examples of the display system according toone aspect of the invention.

[0044]FIG. 13 illustrates a 3D display system including an eye positionfixing device according to one aspect of the invention.

[0045]FIG. 14 illustrates a relationship between the displayed imagesand a viewer's eyes.

[0046]FIG. 15 illustrates a 3D image display system according to oneaspect of the invention.

[0047]FIG. 16A illustrates an exemplary flowchart for explaining theoperation of the system of FIG. 15.

[0048]FIG. 17 is a conceptual drawing for explaining the operation ofthe display device of FIG. 15.

[0049]FIG. 18 illustrates a 3D image display system according to anotheraspect of the invention.

[0050]FIG. 19 illustrates an exemplary flowchart for explaining theoperation of the system of FIG. 18.

[0051]FIG. 20 illustrates an exemplary flowchart for explaining theoperation of the system of FIG. 18.

[0052]FIG. 21A illustrates an eye lens motion detection device.

[0053]FIG. 21B is a conceptual drawing for explaining the movement ofthe eye lenses.

[0054]FIG. 22 is a conceptual drawing for explaining the movement of thecenter points of the displayed images.

[0055]FIG. 23 illustrates a camera system for a 3D display systemaccording to one aspect of the invention.

[0056]FIG. 24 illustrates a display system corresponding to the camerasystem shown in FIG. 23.

[0057]FIG. 25 illustrates an exemplary flowchart for explaining theoperation of the camera and display systems shown in FIGS. 23 and 24.

[0058]FIG. 26A is a conceptual drawing that illustrates parameters for aset of stereoscopic cameras.

[0059]FIG. 26B is a conceptual drawing that illustrates parameters for aviewer's eyes.

[0060]FIG. 27 is a conceptual drawing that illustrates the movement of a: set of stereoscopic cameras.

[0061]FIG. 28 is a conceptual drawing for explaining the eye lensmovement according to the distance between the viewer and an object FIG.29 illustrates a 3D display system for controlling a set of stereoscopiccameras according to another aspect of the invention.

[0062]FIG. 30 illustrates an exemplary block diagram of the cameracontrollers shown in FIG. 29.

[0063]FIG. 31 illustrates an exemplary flowchart for explaining theoperation of the camera controllers according to one aspect of theinvention.

[0064]FIG. 32A illustrates an exemplary table for controlling horizontaland vertical motors.

[0065]FIG. 32B illustrates a conceptual drawing that explains motion ofthe camera.

[0066]FIG. 33 illustrates an exemplary flowchart for explaining theoperation of the system shown in FIG. 29.

[0067]FIG. 34 illustrates a stereoscopic camera controller system usedfor a 3D display system according to another aspect of the invention.

[0068]FIG. 35 illustrates an exemplary table showing the relationshipbetween camera adjusting values and selected cameras.

[0069]FIG. 36A is a top plan view of the plural sets of stereoscopiccameras.

[0070]FIG. 36B is a front elevational view of the plural sets ofstereoscopic cameras.

[0071]FIG. 37 illustrates an exemplary flowchart for explaining theoperation of the system shown in FIG. 34.

[0072]FIG. 38 illustrates a 3D display system according to anotheraspect of the invention.

[0073]FIG. 39 illustrates one example of a 3D display image.

[0074] FIGS. 40A-40H illustrate conceptual drawings that explain therelationship between the 3D mouse cursors and eye lens locations.

[0075]FIG. 41 illustrates an exemplary block diagram of the displaydevices as shown in FIG. 38.

[0076]FIG. 42 illustrates an exemplary flowchart for explaining theoperation of the display devices of FIG. 41.

[0077] FIGS. 43A-43C illustrate conceptual drawings that explain amethod for calculating the location of the center points of the eye lensand the distance between two locations.

[0078]FIG. 44 is a conceptual drawing for explaining a determinationmethod of the location of the center points of the displayed images.

[0079]FIG. 45 illustrates a 3D display system according to anotheraspect of the invention.

[0080]FIG. 46 illustrates an exemplary block diagram of the displaydevice of FIG. 45.

[0081]FIG. 47 is a conceptual drawing for explaining the camera controlbased on the movement of the eye lenses.

[0082]FIG. 48 illustrates an exemplary flowchart for explaining theoperation of the system shown in FIG. 45.

[0083]FIG. 49 illustrates a 3D display system according to anotheraspect of the invention.

[0084]FIG. 50 illustrates an exemplary block diagram of the cameracontroller of FIG. 49.

[0085]FIG. 51 illustrates an exemplary flowchart for explaining thecamera controller of FIG. 50.

[0086]FIG. 52 illustrates an exemplary table for explaining therelationship between the space magnification and camera distance.

[0087]FIG. 53 illustrates an exemplary flowchart for explaining theoperation of the entire system shown in FIG. 49.

[0088]FIG. 54 illustrates a 3D display system according to anotheraspect of the invention.

[0089]FIG. 55 illustrates an exemplary table for explaining therelationship between the camera motion and display angle.

[0090]FIG. 56 illustrates an exemplary flowchart for explaining theentire operation of the system shown in FIG. 54.

[0091]FIG. 57 illustrates a 3D display system according to anotheraspect of the invention.

[0092]FIG. 58 illustrates an exemplary block diagram of the displaydevice of FIG. 57.

[0093]FIGS. 59A and 59B are conceptual drawings for explaining theadjustment of the displayed image.

[0094]FIG. 60 illustrates an exemplary flowchart for explaining theoperation of the system of FIG. 54.

[0095]FIG. 61 illustrates an exemplary block diagram for transmittingstereoscopic images and photographing ratios for the images.

[0096]FIG. 62 illustrates an exemplary block diagram for storing on apersistent memory stereoscopic images and photographing ratios for theimages.

[0097]FIG. 63 illustrates an exemplary format of the data that arestored in the recording medium of FIG. 62.

[0098]FIG. 64 illustrates an exemplary block diagram of a pair ofportable communication devices comprising a pair of digital cameras anda pair of display screens.

[0099]FIG. 65 illustrates an exemplary block diagram of a portablecommunication device for displaying stereoscopic images based on aphotographing ratio and a screen ratio.

[0100]FIGS. 66A and 66B illustrate an exemplary block diagram of aportable communication device for controlling the location of thestereoscopic images.

[0101]FIG. 67 illustrates an exemplary block diagram of a portablecommunication device for controlling space magnification forstereoscopic images.

[0102]FIG. 68 illustrates a conceptual drawing for explaining a portablecommunication device having separate display screens.

[0103]FIGS. 69A and 69B illustrate an exemplary block diagram forexplaining the generation of the stereoscopic images fromthree-dimensional structural data.

[0104] FIGS. 70 illustrates a 3D display system for conforming theresolution between the stereoscopic cameras and display devices.

DETAILED DESCRIPTION OF CERTAIN EMBODIMENTS OF THE INVENTION

[0105]FIG. 1A illustrates one typical 3-D image generating andreproducing apparatus. The system of FIG. 1A uses two display devices soas to display stereoscopic images. The apparatus includes a set ofstereoscopic cameras 110 and 120, spaced apart a predetermined distancefrom each other. The cameras 110 and 120 may be spaced apart as the sameas exists distance between a viewer's two eyes, for photographing anobject 100 at two different positions. Each camera 110 and 120 provideseach photographed image simultaneously or sequentially to the displaydevices 140 and 150, respectively. The display devices 140 and 150 arelocated such that a viewer can watch each image displayed in the devices140 and 150 through their left and right eyes, respectively. The viewercan recognize a 3-D image by simultaneously or sequentially perceivingand synthesizing the left and right images. That is, when the viewersees a pair of stereoscopic images with each eye, a single image(object) is perceived having a 3D quality.

[0106]FIG. 1B illustrates another typical 3-D image generating andreproducing apparatus. The system of FIG. 1B uses one display device soas to display stereoscopic images. The apparatus includes a set ofstereoscopic cameras 110 and 120, spaced apart a predetermined distancefrom each other for photographing the same object 100 at the twodifferent positions. Each camera 110 and 120 provides each photographedimage to a synthesizing device 130. The synthesizing device 130 receivestwo images from the left and right cameras 11O and 120, and sequentiallyirradiates the received images on a display device 160. The synthesizingdevice 130 may be located in either a camera site or a display site. Theviewer wears special glasses 170 that allow each displayed image to beseen by each eye. The glasses 170 may include a filter or a shutter thatallows the viewer to see each image alternately. The display device 160may comprise a LCD or a 3-D glasses such as a head mounted display(HMD). Thus, the viewer can recognize a 3-D image by sequentiallyperceiving the left and right images through each eye.

[0107] Here, according to the distance between the two cameras and theobject to be photographed by the cameras, and the size of thephotographed object, the size of the displayed image is determined.Also, as the distance between the left and right images displayed on thedisplay device has the same ratio as the distance between a viewer's twoeyes, the viewer feels a sense of viewing the actual object in3-dimensions.

[0108] In the above technology, an object may be photographed by acamera while the object moves, the camera moves, or a magnifying(zoom-in) or reducing (zoom-out) imaging function is performed withrespect to the object, not being in a state in which a fixed object isphotographed by a fixed camera. In those situations, the distancebetween the camera and the photographed object, or the size of thephotographed object changes. Thus, a viewer may perceive the imagehaving a sense of distance different than is the actual distance fromthe camera to the object.

[0109] Also, even when the distance between the object and thestereoscopic cameras is fixed during photographing, each viewer hastheir own unique eye distance, a biometric which is measured as thedistance between the center points of the viewer's eyes. For example,the distance between an adult's eyes is quite different from that of achild's eyes. Also the eye distance varies between viewers of the sameage. In the meantime, in current 3D display systems, the distancebetween the center points of each stereoscopic image is fixed at thedistance value of the average adult (i.e., 70 mm) as exemplified inFIGS. 1A and 1B. However, as discussed above, each viewer has their ownpersonal eye distance. This may cause a headache when the viewer seesstereoscopic images as well as the sense of 3-dimensions beingdistorted. In certain instances, the sense of 3-dimensions is not evenperceived.

[0110] In order to display a realistic 3D image, one aspect of theinvention is to adjust display images or display devices such that ascreen ratio (D:E:F) in the display device is substantially the same asa photographing ratio (A:B:C) in the camera. Hereinafter, the term 3Dimages and stereoscopic images will be used to convey the same meaning.Also, a stereoscopic image comprises a pair of two-dimensional planeimages produced by a pair of stereoscopic images. Stereoscopic imagescomprise a plurality of stereoscopic images.

PHOTOGRAPHING RATIO (A:B:C) and SCREEN RATIO (D:E:F)

[0111]FIGS. 2A and 2B illustrate a photographing ratio of a camera. Theratio relates to a scope or the size of the space, being proportional toa range which is seen through a viewfinder of a camera, that the cameracan photograph in a scene. The photographing ratio includes threeparameters (A, B, C). Parameters A and B are defined as horizontal andvertical lengths of the space, respectively, including the object 22photographed by the camera 20. Parameter C is defined as theperpendicular distance between the camera 20 and the object 22.Generally, a camera has its own horizontal and vertical ranges that canphotograph an object, and the ratio of the horizontal and verticallengths is typically constant, e.g., 4:3 or 16:9. Thus, once one of thehorizontal and vertical lengths is determined, the other length may beautomatically determined. In one embodiment of the invention, the camera20 comprises a video camera, a still camera, an analog camera, or adigital camera.

[0112] For the purpose of the explanation, assume that the object 22 islocated “20 m” away from the camera 20 and is photographed such that theobject 22 is included in a single film or an image frame as shown inFIGS. 2A and 2B. If the horizontal distance (A) is 20 m, the verticaldistance (B) would be “15 m” for a 4:3 camera ratio. Since the distancebetween the camera 20 and the object 22 is 10 m, the photographing ratiois 20:15:10=2:1.5:1. In one embodiment of the invention, the presentphotographing ratio while photographing an object may be determinedbased on the optical property of a camera object lens, e.g., the maximumphotographing ratio and minimum photographing ratio.

[0113]FIGS. 3A and 3B illustrate a screen ratio of a display device thatdisplays a photographed image. The screen ratio relates to a range orscope that a viewer can see through a display device. The screen ratioincludes three parameters (D, E, F). Parameters D and E are defined ashorizontal and vertical lengths of the image displayed in the displaydevice 24, respectively. Parameter F is defined as the perpendiculardistance between the display device and a viewer's eye 26. Forconvenience, only one eye 26 and one display device 24 are illustratedinstead of two eyes and a set of display devices in FIGS. 3A and 3B. Fmay be automatically measured using a distance detection sensor or maybe manually measured, or may be fixed. In one embodiment of theinvention, parameters D and E are adjusted such that the photographingratio (A:B:C) equals the screen ratio (D:E:F). Thus the size of theadjusted image in the display device 24 corresponds to that of the imagethat has been captured by the camera 20. This means that a viewerwatches the display image at the same rate the camera 20 photographs anobject. Thus, by always maintaining the relationship of the adjustmentof being “A:B:C=D:E:F” provides a more realistic 3D image to the viewer.Thus, by one embodiment of the invention, if the camera photographs anobject with a large photographing ratio, the image is displayed using alarge screen ratio.

[0114]FIG. 4A illustrates the variation of the distance between anobject lens and a film according to the variation of a focal length ofthe camera 20. (Note that although the term “film” is used in thisspecification, the term is not limited to analog image recording media.For instance, a CCD device or CMOS image sensor may be use to capture animage in a digital context. The camera 20 may have more focal lengthranges, but only four focal length ranges are exemplified in FIG. 4A.

[0115] As shown in FIG. 4A, the distance between a film and an objectlens ranges from d1-d4 according to the focal length of the camera 20.The focal length may be adjusted by a focus adjusting portion (whichwill be explained below) of the camera 20. The distance (d1) is shortestwhen the focal length is “infinity” (∞). When the camera 20 is set tohave an infinity focal length, the camera 20 receives the most amount oflight through the object lens. The distance (d4) is longest when thefocal length is “0.5 m,” where the camera receives the least amount oflight through the object lens. That is, the amount of light coming intothe camera 20 varies according to the focal length of the camera 20.

[0116] Since the location of the object lens is normally fixed, in orderto change the distance from d1 to d4, the location of the film rangesfrom P_(s) to P_(l) as much as “d” according to the focal length. Thefocus adjusting portion of the camera 20 adjusts the location of thefilm from P_(s) to P_(l). The focus adjusting of the camera 20 may bemanually performed or may be automatically made.

[0117]FIG. 4B illustrates the variation of a photographing ratioaccording to the variation of the focal length of the camera 20. Thephotographing ratio (A:B:C) may be expressed as (A/C: B/C). When thecamera is set to have an infinity focal length, the value A/C or B/C isthe biggest amount, which is shown as “2.0/1” in FIG. 4B. In contrast,when the camera 20 is set to have, e.g., a “0.5 m” focal length, thevalue A/C or B/C is the smallest amount, which is shown as “1.0/1” inFIG. 4B. That is, the more amount of light the camera receives, thelarger the photographing ratio. Similarly, the longer the focal length,the greater the photographing ratio.

[0118]FIG. 4C shows the relationship between a photographing ratio and afocal length of a camera. The focal length of the camera may bedetermined, e.g., by detecting a current scale location of the focusadjusting portion of the camera. As shown in FIG. 4C, when the camerahas a focal length range of “0.3 m to infinity,” the focus adjustingportion is located in one position of the scales between 0.3 m andinfinity while the camera is photographing an object. In this situation,the photographing ratio varies linearly as shown in FIG. 4C. If thecamera has a focus adjusting portion that is automatically adjustedwhile photographing an object, the photographing ratio may be determinedby detecting the current focal length that is automatically adjusted.

[0119]FIG. 4D illustrates an exemplary table showing maximum and minimumphotographing ratios of a camera. As described before, a camera has themaximum photographing ratio (A:B:C=3:2:1) when the focal length is thelongest, i.e., a distance of infinity as shown in FIG. 4D. In addition,the camera has the minimum photographing ratio (A:B:C=1.5:1:1) when thefocal length is the shortest, i.e., “0.3 m” as shown in FIG. 4D. Themaximum and minimum photographing ratios of the camera are determined bythe optical characteristic of the camera. In one embodiment, a cameramanufacturing company may provide the maximum and minimum photographingratios in the technical specification of the camera. The table in FIG.4D is used for determining a photographing ratio when the focusadjusting portion is located in one scale between “0.3 m and aninfinity.”

METHOD AND SYSTEM FOR CALCULATING A PHOTOGRAPHING RATIO OF A CAMERA

[0120]FIG. 5A illustrates a photographing ratio calculation apparatusaccording to one aspect of the invention. The apparatus comprises afocus adjusting portion (FAP) 52, a FAP location detection portion 54, amemory 56, and a photographing ratio calculation portion 58. In oneembodiment, the photographing ratio calculation apparatus may beembedded into the camera 20.

[0121] The focus adjusting portion 52 adjusts the focus of the objectlens of the camera 20. The focus adjusting portion 52 may perform itsfunction either manually or automatically. In one embodiment of theinvention, the focus adjusting portion 52 may comprise 10 scales between“0.3 m and infinity,” and is located in one of the scales while thecamera 20 is photographing an object. In one embodiment of theinvention, the focus adjusting portion 52 may use a known focusadjusting portion that is used in a typical camera.

[0122] The FAP location detection portion 54 detects the current scalelocation of the focus adjusting portion 52 among the scales. In oneembodiment of the invention, the FAP location detection portion 54 maycomprise a known position detection sensor that detects the scale valuein which the focus adjusting portion 52 is located. In anotherembodiment of the invention, since the variation of the scale locationis proportional to the distance between the object lens and film asshown in FIG. 4A, the FAP location detection portion 54 may comprise aknown distance detection sensor that measures the distance between theobject lens and film.

[0123] The memory 56 stores data representing maximum and minimumphotographing ratios of the camera 20. In one embodiment of theinvention, the memory 56 comprise a ROM, a flash memory or aprogrammable ROM. This may apply to all of the other memories describedthroughout the specification.

[0124] The photographing ratio calculation portion 58 calculates aphotographing ratio (A:B:C) based on the detected scale location and themaximum and minimum photographing ratios. In one embodiment of theinvention, the photographing ratio calculation portion 58 comprises adigital signal processor (DSP) calculating the ratio (A:B:C) using thefollowing $\begin{matrix}{{{Equation}\quad \text{I:}}{A = {{\left( \frac{A_{\max} - A_{\min}}{c} \right) \times \left( \frac{S_{cur}}{S_{tot}} \right)} + \frac{A_{\min}}{c}}}{{Equation}\quad \text{II:}}} & \quad \\{B = {{\left( \frac{B_{\max} - B_{\min}}{c} \right) \times \left( \frac{S_{cur}}{S_{tot}} \right)} + \frac{B_{\min}}{c}}} & \quad\end{matrix}$

[0125] In Equations I and II, parameters A_(max) and B_(max) representhorizontal and vertical length values (A and B) of the maximumphotographing ratio, respectively, exemplified as “3” and “2” in FIG.4D. Parameters A_(min) and B_(min) represent horizontal and verticallength values (A and B) of the minimum photographing ratio,respectively, shown as “1.5” and “1” in FIG. 4D. Parameters S_(cur) andS_(tot) represent the current detected scale value and the total scalevalue, respectively. Parameter “c” represents the distance value of themaximum or minimum photographing ratio. Since the photographing ratio(A:B:C) represents the relative proportion between the three parameters,A, B and C, the parameters may be simplified as shown in FIG. 4D. Forexample, the photographing ratio, A:B:C=300:200:100, is the same asA:B:C=3:2:1. In one embodiment of the invention, the parameter “c” hasthe value “1” as shown in FIG. 4D.

[0126] In another embodiment of the invention, the photographing ratiocalculation portion 58 calculates a photographing ratio (A:B:C) suchthat the ratio falls between the maximum and minimum photographingratios and at the same time is proportional to the value of the detectedscale location. Thus, as long as the ratio falls between the maximum andminimum photographing ratios and is proportional to the value of thedetected scale location, any other equation may be used for calculatingthe photographing ratio.

[0127] Referring to FIG. 6A, the operation of the photographing ratiocalculation apparatus of FIG. 5A will be explained. The camera 20photographs an object (602). In one embodiment of the invention, thecamera 20 comprise a single (mono) camera. In another embodiment of theinvention, the camera 20 comprise a pair of stereoscopic cameras asshown in FIG. 1A. In either case, the operation of the apparatus will bedescribed based on the single camera for convenience.

[0128] Maximum and minimum photographing ratios are provided from thememory 56 to the photographing ratio calculation portion 58 (604). Inone embodiment of the invention, the photographing ratio calculationportion 58 may store the maximum and minimum photographing ratiostherein. In this situation, the memory 56 may be omitted from theapparatus.

[0129] The FAP location detection portion 54 detects the currentlocation of the focus adjusting portion 52 while the camera 20 isphotographing the object (606). While the camera is photographing theobject, the focal length may be changed. The detected current locationof the focus adjusting portion 52 is provided to the photographing ratiocalculation portion 58.

[0130] The photographing ratio calculation portion 58 calculates ahorizontal value (A) of a current photographing ratio from Equation I(608). It is assumed that the detected current location value is “5”among the total scale values “10.” Using Equation I and the table ofFIG. 4D, the horizontal value A is obtained as follows.$A = {{{\left( \frac{A_{\max} - A_{\min}}{c} \right) \times \left( \frac{S_{cur}}{S_{tot}} \right)} + \frac{A_{\min}}{c}} = {{{\left( \frac{3 - 1.5}{1} \right) \times \left( \frac{5}{10} \right)} + \frac{1.5}{1}} = 2.25}}$

[0131] The photographing ratio calculation portion 58 calculates avertical value (B) of a current photographing ratio from Equation II(610). In the above example, using Equation II and the table of FIG. 4D,the vertical value B is obtained as follows.$B = {{{\left( \frac{B_{\max} - B_{\min}}{c} \right) \times \left( \frac{S_{cur}}{S_{tot}} \right)} + \frac{B_{\min}}{c}} = {{{\left( \frac{2 - 1}{1} \right) \times \left( \frac{5}{10} \right)} + \frac{1}{1}} = 1.5}}$

[0132] The photographing ratio calculation portion 58 retrievesparameter C from the maximum and minimum ratios that have been used forcalculating parameters A and B (612). Referring to the table of FIG. 4D,the distance value (C) is “1.” The photographing ratio calculationportion 58 provides a current photographing ratio (A:B:C) (614). In theabove example, the current photographing ratio=2.25:1.5:1.

[0133]FIG. 5B illustrates a block diagram of a photographing ratiocalculation apparatus according to another aspect of the invention. Theapparatus comprises an iris 62, an iris opening detection portion 64, amemory 66 and a photographing ratio calculation portion 68. In oneembodiment of the invention, the photographing ratio calculationapparatus is embedded into the camera 20.

[0134] The iris 62 is a device that adjusts an amount of light cominginto the camera 20 according to the degree of its opening. When thedegree of the opening of the iris 62 is largest, the maximum amount oflight shines on the film of the camera 20. This largest openingcorresponds to the longest focal length and the maximum photographingratio. In contrast, when the degree of the opening of the iris 62 issmallest, the least amount of light comes into the camera 20. Thissmallest opening corresponds to the shortest focal length and theminimum photographing ratio. In one embodiment of the invention, theiris 62 may be a known iris that is used in a typical camera.

[0135] The iris opening detection portion 64 detects the degree of theopening of the iris 62. The degree of the opening of the iris 62 may bequantitized to a range of, for example, 1-10. Degree “10” may mean thelargest opening of the iris 62 and degree “1” may mean the smallestopening of the iris 62. The memory 66 stores data representing maximumand minimum photographing ratios of the camera 20.

[0136] The photographing ratio calculation portion 68 calculates aphotographing ratio (A:B:C) based on the detected degree of the openingand the maximum and minimum photographing ratios. In one embodiment ofthe invention, the photographing ratio calculation portion 68 comprisesa digital signal processor (DSP) calculating the ratio (A:B:C) using thefollowing Equations III and IV. $\begin{matrix}{{{Equation}\quad \text{III:}}{A = {{\left( \frac{A_{\max} - A_{\min}}{c} \right) \times \left( \frac{I_{cur}}{I_{largest}} \right)} + \frac{A_{\min}}{c}}}{{Equation}\quad \text{IV:}}} & \quad \\{B = {{\left( \frac{B_{\max} - B_{\min}}{c} \right) \times \left( \frac{I_{cur}}{I_{largest}} \right)} + \frac{B_{\min}}{c}}} & \quad\end{matrix}$

[0137] In Equations III and IV, parameters A_(max) and B_(max), A_(min)and B_(min), and “c” are the same as the parameters used in Equations Iand II. Parameters I_(cur) and I_(largest) represent the detectedcurrent degree of the opening and the largest degree of the opening,respectively.

[0138] Referring to FIG. 6B, the operation of the photographing ratiocalculation apparatus will be described. The operation with regard tothe first two procedures 702 and 704 is the same as those in FIG. 6A.

[0139] The iris opening detection portion 64 detects the current degreeof the opening of the iris 62 while the camera 20 is photographing theobject (706). The detected degree of the opening of the iris 62 isprovided to the photographing ratio calculation portion 68.

[0140] The photographing ratio calculation portion 68 calculates ahorizontal value (A) of a current photographing ratio from Equation III(708). It is assumed that the detected current opening degree is 2 amongthe total degree values 10. Using Equation III and FIG. 4D, thehorizontal value A is obtained as follows.$A = {{{\left( \frac{A_{\max} - A_{\min}}{c} \right) \times \left( \frac{I_{cur}}{I_{largest}} \right)} + \frac{A_{\min}}{c}} = {{{\left( \frac{3 - 1.5}{1} \right) \times \left( \frac{2}{10} \right)} + \frac{1.5}{1}} = 1.8}}$

[0141] The photographing ratio calculation portion 68 calculates avertical value (B) of a current photographing ratio from Equation IV(710). In the above example, using equation IV and FIG. 4D, the verticalvalue B is obtained as follows.$B = {{{\left( \frac{B_{\max} - B_{\min}}{c} \right) \times \left( \frac{I_{cur}}{I_{largest}} \right)} + \frac{B_{\min}}{c}} = {{{\left( \frac{2 - 1}{1} \right) \times \left( \frac{2}{10} \right)} + \frac{1}{1}} = 1.2}}$

[0142] The photographing ratio calculation portion 68 retrievesparameter C from the maximum and minimum ratios that have been used forcalculating parameters A and B (712). Referring to FIG. 4D, the distancevalue is “1.” The photographing ratio calculation portion 68 provides acurrent photographing ratio (A:B:C) (714). In the above example, acurrent photographing ratio is 1.8:1.2:1.

[0143]FIG. 7 illustrates a camera comprising the photographing ratiocalculation apparatus as shown in FIGS. 5A and 5B. The camera 20comprises an image data processing apparatus 70, a microcomputer 72, aphotographing ratio calculation apparatus 74, and a data combiner 76.

[0144] In one embodiment of the invention, the camera 20 comprises ananalog camera and a digital camera. When the camera 20 photographs anobject, the image data processing apparatus 70 performs a typical imageprocessing of the photographed image according to the control of themicrocomputer 72. In one embodiment of the invention, the image dataprocessing apparatus 70 may comprise a digitizer that digitizes thephotographed analog image into digital values, a memory that stores thedigitized data, and a digital signal processor (DSP) that performs animage data processing of the digitized image data (all not shown). Theimage data processing apparatus 70 provides the processed data to a datacombiner 76.

[0145] In one embodiment, the photographing ratio calculation apparatus74 comprises the apparatus shown in FIGS. 5A or 5B. The photographingratio calculation apparatus 74 calculates a photographing ratio (A:B:C).The calculated photographing ratio (A:B:C) data are provided from theapparatus 74 to the data combiner 76.

[0146] The microcomputer 72 controls the image data processing apparatus70, the photographing ratio calculation apparatus 74, and the datacombiner 76 such that the camera 20 outputs the combined data 78. In oneembodiment of the invention, the microcomputer 72 controls the imagedata processing apparatus 70 such that the apparatus properly processesthe digital image data. In this embodiment of the invention, themicrocomputer 72 controls the photographing ratio calculation apparatus74 to calculate a photographing ratio for the image being photographed.In this embodiment of the invention, the microcomputer 72 controls thedata combiner 76 to combine the processed data and the photographingratio data corresponding to the processed data. In one embodiment of theinvention, the microcomputer 72 may provide a synchronization signal tothe data combiner 76 so as to synchronize the image data and the ratiodata. As discussed above, as long as the current scale location of thefocus adjusting portion or the opening degree of the iris is notchanged, the photographing ratio is not changed. The microcomputer 72may detect the change of the scale location or the opening degree, andcontrol the data combiner 76 such that the image data and thecorresponding ratio data are properly combined.

[0147] In one embodiment of the invention, the microcomputer 72 isprogrammed to perform the above function using typical microcomputerproducts, available from the Intel, IBM and Motorola companies, etc.This product may also apply to the other microcomputers describedthroughout this specification.

[0148] The data combiner 76 combines the image data from the image dataprocessing apparatus 70 and the calculated photographing ratio (A:B:C)data according to the control of the microcomputer 72. The combiner 76outputs the combined data 78 in which the image data and the ratio datamay be synchronized with each other. In one embodiment of the invention,the combiner 76 comprises a known multiplexer.

METHOD AND SYSTEM FOR CONTROLLING A SCREEN RATIO BASED ON APHOTOGRAPHING RATIO

[0149]FIG. 8 illustrates a system for displaying stereoscopic imagessuch that a photographing ratio (A:B:C) is substantially the same as ascreen ratio (D:E:F). The system comprises a camera site 80 and adisplay site 82. The camera site 80 transmits a photographing ratio(A:B:C) and photographed image to the display site 82. The display site82 displays the transmitted image such that a screen ratio (D:E:F) issubstantially the same as the photographing ratio (A:B:C). In oneembodiment of the invention, the camera site 80 may comprise a singlecamera and the display site may comprise a single display device. Inanother embodiment of the invention, the camera site may comprise a setof stereoscopic cameras and the display site may comprise a set ofdisplay devices as shown in FIG. 8.

[0150] The embodiment of camera site 80 shown in FIG. 8 comprises a setof stereoscopic cameras 110 and 120, and transmitters 806 and 808. Thestereoscopic left and right cameras 110 and 120 may be located as shownin FIG. 1A with regard to an object to be photographed. The cameras 110and 120 comprise the elements described with respect to FIG. 7. Each ofthe cameras 110 and 120 provides its own combined data 802 and 804 tothe transmitters 806 and 808, respectively. Each transmitter 806 and 808transmits the combined data 802 and 804 to the display site 82 through anetwork 84. The network 84 may comprise a wire transmission or awireless transmission. In one embodiment of the invention, eachtransmitter 806 and 808 is separate from the cameras 110 and 120. Inanother embodiment of the invention, each transmitter 806 and 808 may beembedded into each camera 110 and 120. For convenience, it is assumedthat both of the photographing ratios are referred to as “A1:B1:C1” and“A2:B2:C2,” respectively.

[0151] In one embodiment of the invention, the photographing ratios“A1:B1:C1” and “A2:B2:C2” are substantially the same. In one embodimentof the invention, the data 802 and 804 may be combined and transmittedto the display site 82. In one embodiment of the invention, thephotographing ratio may have a standard data format in each of thecamera and display sites so that the display site can identify thephotographing ratio easily.

[0152] The display site 82 comprises a set of receivers 820, 832, a setof display devices 86, 88. Each receiver 820, 832 receives the combineddata transmitted from the camera site 80 and provides each data set tothe display devices 86, 88, respectively. In one embodiment of theinvention, each of the receivers 820, 832 is separate from the displaydevices 86, 88. In another embodiment of the invention, receivers 820,832 may be embedded into each display device 86, 88.

[0153] The display devices 86 and 88 comprise data separators 822 and834, image size adjusting portions 828 and 840, and display screens 830and 842. The data separators 822 and 834 separate the photographingratio data (824, 838) and the image data (826, 836) from the receiveddata. In one embodiment of the invention, each of the data separators822 and 834 comprises a typical demultiplexer.

[0154] The image size adjusting portion 828 adjusts the size of theimage to be displayed in the display screen 830 based on thephotographing ratio (A1:B1:C1), and screen-viewer distance (F1) anddisplay screen size values (G1, H1). The screen-viewer distance (F1)represents the distance between the display screen 830 and one of aviewer's eyes, e.g., a left eye, that is directed to the screen 830. Inone embodiment of the invention, the distance F1 may be fixed. In thissituation, a viewer's eyes may be located in a eye fixing structure,which will be described in more detail later. Also, the image sizeadjusting portion 828 may store the fixed value F1 therein. The screensize values G1 and H1 represent the horizontal and vertical dimensionsof the screen 830, respectively. In one embodiment of the invention, thesize values G1 and H1 may be stored in the image size adjusting portion828.

[0155] The image size adjusting portion 840 adjusts the size of theimage to be displayed in the display screen 842 based on thephotographing ratio (A2:B2:C2), and screen-viewer distance (F2) anddisplay screen size values (G2, H2). The screen-viewer distance (F2)represents the distance between the display screen 842 and one of aviewer's eyes, e.g., a right eye, that is directed to the screen 842. Inone embodiment of the invention, the distance F2 may be fixed. In oneembodiment of the invention, the screen-viewer distance (F2) issubstantially the same as the screen-viewer distance (F1). The screensize values G2 and H2 represent the horizontal and vertical dimensionsof the screen 842, respectively. In one embodiment of the invention, thedisplay screen size values G2 and H2 are substantially the same as thedisplay screen size values G1 and H1.

[0156] The operation of the image size adjusting portions 828 and 840will be described in more detail by referring to FIGS. 9 and 10. Sincethe operations of the two image size adjusting portions 828 and 840 aresubstantially the same, for convenience, only the operation with regardto the image size adjusting portion 828 will be explained.

[0157] The image data 826, the photographing ratio data (A1:B1:C1) andthe screen-viewer distance (F1) are provided to the image size adjustingportion 828 (902). A screen ratio (D1:E:F1) is calculated based on thephotographing ratio (A1:B:C1) and the screen-viewer distance (F1) usingthe following Equation V (904). Since the value F1 is already provided,the parameters D1 and E1 of the screen ratio are obtained from EquationV. Equation  V: A1 : B1 : C1 = D1 : E1 : F1${D1} = {{A1} \times \frac{F1}{C1}}$${E1} = {{B1} \times \frac{F1}{C1}}$

[0158] The horizontal and vertical screen size values (G1, H1) of thedisplay screen 830 are provided to the image size adjusting portion 828(906). In one embodiment of the invention, the screen size values G1 andH1, and the distance value F1 are fixed and stored in the image sizeadjusting portion 828. In another embodiment of the invention, thescreen size values G1 and H1, and the distance value F1 are manuallyprovided to the image size adjusting portion 828.

[0159] Image magnification (reduction) ratios d and e are calculatedfrom the following Equation VI (908). The ratios d and e representhorizontal and vertical magnification (reduction) ratios for the displayscreens 830 and 842, respectively. Equation  VI: $\begin{matrix}\begin{matrix}{d = \frac{D1}{G1}} \\{e = \frac{E1}{H1}}\end{matrix} & \quad\end{matrix}$

[0160] This is to perform magnification or reduction of the providedimage 826 with regard to the screen sizes (G1, H1). If the calculatedvalue “D1” is greater than the horizontal screen size value (G1), theprovided image needs to be magnified as much as “d.” If the calculatedvalue “D1” is less than the horizontal screen size value (G1), theprovided image needs to be reduced as much as “d.” The same applies tothe calculated value “E1.” This magnification or reduction enables aviewer to recognize the image at the same ratio that the camera 110photographed the object. The combination of the display devices 86 and88 provides a viewer with a more realistic three-dimensional image.

[0161] It is determined whether the magnification (reduction) ratios (d,e) are greater than “1” (910). If both of the ratios (d, e) are greaterthan 1, the image data 826 are magnified as much as “d” and “e,”respectively, as shown in FIG. 10A (912). In one embodiment of theinvention, the portion of the image greater than the screen sizes (G1,H1) is cut out as shown in FIG. 10A (914).

[0162] If both of the ratios “d” and “e” are not greater than 1, it isdetermined whether the magnification (reduction) ratios (d, e) are lessthan “1” (916). If both of the ratios d and e are less than 1, the imagedata 826 are reduced as much as “d” and “e,” respectively, as shown inFIG. 10B (918). In one embodiment of the invention, the blank portion ofthe screen is filled with background color, e.g., black color, as shownin FIG. 10B (920).

[0163] If both of the ratios d and e are equal to 1, no adjustment ofthe image size is made (922). In this situation, since the magnification(reduction) ratio is 1, no magnification or reduction of the image ismade as shown in FIG. 10C.

[0164] Now referring to FIG. 11, the entire operation of the systemshown in FIG. 8 will be described. Photographing an object is performedusing a set of stereoscopic cameras 110 and 120 (1120), as exemplifiedin FIG. 1A. Each of the cameras 110 and 120 calculates the photographingratio (A1:B1:C1) and (A2:B2:C2), respectively (1140), for example, usingthe method shown in FIG. 6.

[0165] The image data and the photographing ratio that are calculatedfor the image are combined for each of the stereoscopic cameras 110 and120 (1160). The combined data are illustrated as reference numerals 802and 804 in FIG. 8. In one embodiment of the invention, the combining isperformed per a frame of the image data. In one embodiment of theinvention, as long as the photographing ratio remains unchanged, thecombining may not be performed and only image data without thephotographing ratio may be transmitted to the display site 82. In thatsituation, when the photographing ratio is changed, the combining mayresume. Alternatively, the photographing ratio is not combined, andrather, is transmitted separately from the image data. Each of thetransmitters 806 and 808 transmits the combined data to the display site82 through the communication network 84 (1180).

[0166] Each of the receivers 820 and 832 receives the transmitted datafrom the camera site 80 (1200). The photographing ratio and image dataare separated from the combined data (1220). Alternatively to 1200 and1220, the image data and photographing ratio are separately received asthey are not combined in transmission. In one embodiment of theinvention, the combined data may not include a photographing ratio. Inthat circumstance, the photographing ratio that has been received mostrecently is used for calculating the screen ratio. In one embodiment ofthe invention, the screen ratio may remain unchanged until the newphotographing ratio is received.

[0167] The screen ratios (D1:E1:F1) and (D2:E2:F2) for each of thedisplay devices 86 and 88 are calculated using the method described withregard to FIG. 9 (1240). The stereoscopic images are displayed such thateach of the photographing ratios (A1:B1:C1) and (A2:B2:C2) issubstantially the same as each of the screen ratios (D1:E1:F1) and(D2:E2:F2) (1260). In this situation, the image may be magnified orreduced with regard to the screen size of each of the display devices 86and 88 as discussed with reference to FIGS. 9 and 10.

METHOD AND SYSTEM FOR CONTROLLING THE DISPLAY LOCATION OF A STEREOSCOPICIMAGE

[0168]FIG. 12 illustrates examples of the display system according toone embodiment of the invention. FIG. 12A illustrates a head mountdisplay (HMD) system. The HMD system comprises the pair of the displayscreens 1200 and 1220. For convenience, the electronic display mechanismas exemplified in FIG. 8 is omitted in this HMD system. A viewer wearsthe HMD on his or her head and watches stereoscopic images through eachdisplay screen 1200 and 1220. Thus, in one embodiment of the invention,the screen-viewer's eye distance (F) may be fixed. In another embodimentof the invention, the distance (F) may be measured with a known distancedetection sensor and provided to the HMD system. Another embodiment ofthe invention includes a 3D display system as shown in FIG. 1B. Anotherembodiment of the display devices includes a pair of projection devicesthat project a set of stereoscopic images on the screen.

[0169]FIG. 12B illustrates a 3D display system according to anotherembodiment of the invention. The display system comprises a V shapedmirror 1240, and a set of display devices 1260 and 1280. In oneembodiment of the invention, the display devices 1260 and 1280 aresubstantially the same as the display devices 86 and 88 of FIG. 8 exceptfor further comprising an inverting portion (not shown), respectively.The inverting portion inverts the left and right sides of the image tobe displayed. The V shaped mirror 1240 reflects the images coming fromthe display devices 1260 and 1280 to a viewer's eyes. Thus, the viewerwatches a reflected image from the V shaped mirror 1240. The 3D displaysystem comprising the V shaped mirror is disclosed in U.S. applicationSer. No. 10/067,628, which was filed on Feb. 4, 2002, by the sameinventor as this application and is incorporated by reference herein.For convenience, hereinafter, the description of inventive aspects willbe mainly made based on the display system as shown in FIG. 12B,however, the invention is applicable to other display systems such asthe one shown in FIG. 12A.

[0170]FIG. 13 illustrates a 3D display system including an eye positionfixing device 1300 according to one aspect of the invention. Referringto FIGS. 13A and 13B, the eye position fixing device 1300 is located infront of the V shaped mirror 1240 at a predetermined distance from themirror 1240. The eye position fixing device 1300 is used for fixing thedistance between the mirror 1240 and a viewer's eyes. The eye positionfixing device 1300 is also used for locating a viewer's eyes such thateach of the viewer's eyes are substantially perpendicular to each of themirror (imaginary) images. A pair of holes 1320 and 1340 defined in thedevice 1300 are configured to allow the viewer to see each of the centerpoints of the reflected images. In one embodiment of the invention, thesize of each of the holes 1320 and 1340 is big enough to allow theviewer to see a complete half portion (left or right portion) of the Vshaped mirror 1240 at a predetermined distance and location asexemplified in FIGS. 13A and 13B. In one embodiment of the invention,the eye position fixing device 1300 may be used for fixing the locationof a viewer's eyes as necessary with regard to the other aspects of theinvention as discussed below.

[0171]FIG. 14A illustrates a relationship between the displayed imagesand a viewer's eyes. Distance (W_(d)) represents the distance betweenthe center points (1430, 1440) of each of the displayed images (1410,1420). Distance (W_(a)) represents the distance between the centerpoints (1450, 1460) of each of a viewer's eyes. The distance W_(a)varies from person to person. Normally the distance increases as aperson grows and it does not change when he or she reaches a certainage. The average distance of an adult may be 70 mm. Some people may have80 mm distance, other people may have 60 mm distance. Distance (V_(a))represents the distance between the center points (1470, 1480) of eachof a viewer's eye lenses. Here, a lens means a piece of roundtransparent flesh behind the pupil of an eye. The lens moves along themovement of the eye. The distance V_(a) changes according to thedistance (F) between an object and the viewer's eyes. The farther thedistance (F) is, the greater the value V_(a) becomes. Referring to FIG.14B, when a viewer sees an object farther than, for example, 10,000 m,V_(a) has the maximum value (V_(amax)) which is substantially the sameas the distance W_(a).

[0172] Traditional 3D display systems display images without consideringthe value W_(a). This means that the distance value (W_(d)) is the samefor all viewers regardless of the fact that they have a different W_(a)value. These traditional systems caused several undesirable problemssuch as headache or dizziness of the viewer, and deterioration of asense of three dimension. In order to produce a more realisticthree-dimensional image and to reduce headaches or dizziness of aviewer, the distance W_(d) needs to be determined by considering thedistance W_(a). The consideration of the W_(a) value may provide aviewer with better and more realistic three-dimensional images. In oneembodiment of the invention, the distance W_(d) is adjusted such thatthe distance W_(d) is substantially the same as W_(a).

[0173]FIG. 15 illustrates a 3D image display system according to oneaspect of the invention. Once again, the system may be used with, forexample, either a HMD system or a display system with the V shapedmirror shown in FIGS. 13A and 13B, a projection display system,respectively.

[0174] The system shown in FIG. 15 comprises a pair of display devices1260 and 1280, and a pair of input devices 1400 and 1500. Each of theinput devices 1400 and 1500 provides the distance value W_(a), to eachof the display devices 1260 and 1280. In one embodiment of theinvention, each of the input devices 1400 and 1500 comprises a keyboard,a mouse, a pointing device, or a remote controller. In one embodiment ofthe invention, one of the input devices 1400 and 1500 may be omitted andthe other input device is used for providing the distance value W_(a) toboth of the display devices 1260 and 1280.

[0175] The display devices 1260 and 1280 comprise interfaces 1510 and1550, microcomputers 1520 and 1560, display drivers 1530 and 1570, anddisplay screens 1540 and 1580, respectively. In one embodiment of theinvention, each of the display screens 1540 and 1580 comprises a LCDscreen, a CRT screen, or a PDP screen. The interfaces 1510 and 1550provide the interface between the input devices 1400 and 1500 and themicrocomputers 1520 and 1560, respectively. In one embodiment of theinvention, each of the interfaces 1510 and 1550 comprises a typicalinput device controller and/or a typical interface module (not shown).

[0176] There may be several methods to measure and provide the distance(W_(a)). As one example, an optometrist may measure the W_(a) value of aviewer with eye examination equipment. In this situation, the viewer mayinput the value (W_(a)) via the input devices 1400, 1500. As anotherexample, an eye lens motion detector may be used in measuring the W_(a)value. In this situation, the W_(a) value may be provided from thedetector to either the input devices 1400, 1500 or the interfaces 1510,1550 in FIG. 15.

[0177] As another example, as shown in FIG. 14C, the W_(a) value may bemeasured using a pair of parallel pipes 200, 220, about 1 m in lengthand about 1 mm in diameter, which are spaced approximately 1 cm apartfrom a viewer's eyes. Each end of the pipes 200, 220 is open. The pipedistance (P_(d)) may be adjusted between about 40 mm and about 120 mm bywidening or narrowing the pipes 200, 220. The pipes 200, 220 maintain aparallel alignment while they are widened or narrowed. A ruler 240 maybe attached into the pipes 200, 220, as shown in FIG. 14C so that theruler 240 can measure the distance between the pipes 200, 220. When theviewer sees the holes 260, 280 completely through the holes 200, 220,respectively, the ruler 240 indicates the W_(a) value of the viewer. Inanother embodiment, red and blue color materials (paper, plastic, orglass) may cover the holes 260, 280, respectively. In this situation,the pipe distance (P_(d)) is the W_(a) value of the viewer where theviewer perceives a purple color from the holes 260, 280 by thecombination of the red and blue colors.

[0178] Each of the microcomputers 1520 and 1560 determines an amount ofmovement for the displayed images based on the provided W_(a) value suchthat the W_(d) value is substantially the same as the W_(a) value. Inone embodiment of the invention, each microcomputer (1520, 1560)initializes the distance value W_(d) and determines an amount ofmovement for the displayed images based on the value W_(a) and theinitialized value W_(d). Each of the display drivers 1530 and 1570 movesthe displayed images based on the determined movement amount anddisplays the moved images on each of the display screens 1540 and 1580.In one embodiment of the invention, each microcomputer (1520, 1560) mayincorporate the function of each of the display drivers 1530 and 1570.In that situation, the display drivers 1530 and 1570 may be omitted.

[0179] Referring to FIG. 16, the operation of the system of FIG. 15 willbe described. A set of stereoscopic images are displayed in the pair ofdisplay screens 1540 and 1580 (1610). The stereoscopic images may beprovided from the stereoscopic cameras 110 and 120, respectively, asexemplified in FIG. 1A. The distance (W_(d)) between the center pointsof the displayed images is initialized (1620). In one embodiment of theinvention, the initial value may comprise the eye distance value of theaverage adult, e.g., “70 mm.” The distance (W_(a)) between the centerpoints of a viewer's eye lenses is provided (1630).

[0180] It is then determined whether W_(a) equals W_(d) (1640). If W_(a)equals W_(d), no movement of the displayed images is made (1680). Inthis situation, since the distance (W_(a)) between the center points ofthe viewer's eye is the same as the distance (W_(d)) between the centerpoints of the displayed images, no adjustment of the displayed images ismade.

[0181] If W_(a) does not equal W_(d), it is determined whether W_(a) isgreater than W_(d) (1650). If W_(a) is greater than W_(d), the distance(W_(d)) needs to be increased until W_(d) equals W_(a). In thissituation, the left image 1750 displayed in the left screen 1540 ismoved to the left side and the right image 1760 displayed in the rightscreen 1580 is moved to the right side until the two values aresubstantially the same as shown in FIG. 17A. Referring to FIG. 17B,movements of the displayed images 1750 and 1760 are conceptuallyillustrated for the display system with a V shaped mirror. Since the Vshaped mirror reflects the displayed images, which have been receivedfrom the display devices 1260 and 1280, to a viewer, in order for theviewer to see the adjusted images through the mirror as shown in FIG.17A, the displayed images 1750 and 1760 need to be moved with regard tothe V shaped mirror as shown in FIG. 17B. That is, when the displayedimages 1750 and 1760 are moved as shown in FIG. 17B, the viewer who seesthe V shaped mirror perceives the image movement as shown in FIG. 17A.

[0182] With regard to the HMD system shown in FIG. 12A, the movementdirection of the displayed images is the same as the direction of thoseshown in FIG. 17A. With regard to the projection display systemdescribed in connection with FIG. 15, since the projection displaysystem projects images into a screen that is located across theprojection system, the movement direction of the displayed images isopposite to the direction of those shown in FIG. 17A.

[0183] If it is determined that W_(a) is not greater than W_(d), thedistance W_(d) needs to be reduced until W_(d) equals W_(a). Thus, theleft image 1770 displayed in the display device 1260 is moved to theright side and the right image 1780 displayed in the display device 1280is moved to the left side until the two values are substantially thesame as shown in FIGS. 17C and 17D. The same explanation with regard tothe movement of the displayed images described in FIGS. 17A and 17Bapplies to the system of FIGS. 17C and 17D.

[0184]FIG. 18 illustrates a 3D image display system according to anotherembodiment of the invention. The system comprises an input device 1810,a microcomputer 1820, a pair of servo mechanisms 1830 and 1835, and apair of display devices 1840 and 1845. The input device 1810 provides aviewer's input, i.e., the distance value W_(a), to each of the displaydevices 1260 and 1280. In one embodiment of the invention, the inputdevice 1810 may be a keyboard, a mouse, a pointing device, or a remotecontroller, for example. An interface is omitted for convenience.

[0185] The microcomputer 1820 determines an amount of the movement forthe display devices 1840 and 1845 based on the provided value W_(a) suchthat the W_(d) value is substantially the same as the W_(a) value. Inone embodiment of the invention, the microcomputer 1820 initializes thedistance value (W_(d)) and determines an amount of the movement for thedisplay devices 1840 and 1845 based on the value W_(a) and theinitialized value W_(d). Each of the servo mechanisms 1830 and 1835moves the display devices 1840 and 1845, respectively, based on thedetermined movement amount.

[0186] Referring to FIG. 19, the operation of the system of FIG. 18 willbe described. Each of stereoscopic images is displayed in the displaydevices 1840 and 1845 (1850). The distance (W_(d)) between the centerpoints of the displayed images is initialized (1855). In one embodimentof the invention, the initial value may be “70 mm.” The distance (W_(a))between the center points of a viewer's eyes is provided to themicrocomputer 1820 (1860). It is determined whether W_(a) equals W_(d)(1870). If W_(a) equals W_(d), no movement of the display devices 1840and 1845 is made (1910). If it is determined that W_(a) is greater thanW_(d) (1880), the servo mechanisms 1830 and 1835 move the displaydevices 1840 and 1845, respectively such that W_(d) is widened to W_(a)as shown in FIGS. 20A and 20B. If it is determined that W_(a) is notgreater than W_(d), the servo mechanisms 1830 and 1835 move the displaydevices 1840 and 1845, respectively such that W_(d) is narrowed to W_(a)as shown in FIGS. 20C and 20D.

[0187] In another embodiment of the invention, the distance (V_(a)) isautomatically detected using a known eye lens motion detector. Thisembodiment of the invention will be described referring to FIG. 21A. Thedetector 2100 detects the distance V_(a) between the center points of aviewer's eye lenses. In addition, the detector 2100 detects thelocations of each of the eye lenses. In FIGS. 21A and 21B, A_(2L) andA_(2R) represent the center points of a viewer's eye lenses, and A_(3L)and A_(3R) represent the center points of a viewer's eyes. As seen inFIGS. 21A and 21B, the A_(3L) location is fixed, but the A_(2L) locationmoves. The detector 2100 detects the current locations of each of theeye lenses. In one embodiment of the invention, the detector 2100comprises a known eye lens detecting sensor disclosed, for example, inU.S. Pat. No. 5,526,089.

[0188] The detected distance and location values are provided to amicrocomputer 2120. The microcomputer 2120 receives the distance valueV_(a) and determines an amount of movement for the displayed images oran amount of movement for the display devices similarly as describedwith regard to FIGS. 15-20. The determined amount is used forcontrolling either the movement of the displayed images or the movementof the display devices. In addition, the microcomputer 2120 determinesnew locations of the center points of the images based on the locationvalues of the eye lenses. In this way, the microcomputer 2120 controlsthe display drivers (1530, 1570) or the servo mechanisms (1830, 1835) tomove the stereoscopic images from the current center points 2210 and2230 of the images to, for example, new center points 2220 and 2240 asshown in FIG. 22.

METHOD AND SYSTEM FOR PROVIDING THE MOTION INFORMATION OF STEREOSCOPICCAMERAS

[0189]FIG. 23 illustrates a camera system for a 3D display systemaccording to one aspect of the invention. The camera system is directedto provide photographed image data and camera motion detection data to adisplay site. The camera system comprises a set of stereoscopic cameras2200, 2210, motion detection devices 2220, 2230, combiners 2240, 2250,and transmitters 2280, 2290. Each of the stereoscopic cameras 2200, 2210captures an image and provides the captured image data to each of thecombiners 2240, 2250.

[0190] The motion detection devices 2220 and 2230 detect the motion ofthe cameras 2200 and 2210, respectively. The motion of the cameras 2200and 2210 may comprise motions for upper and lower directions, and leftand right directions as shown in FIG. 23. Each detection device (2220,2230) provides the detection data to each of the combiners 2240 and2250. In one embodiment of the invention, if each of the detectiondevices 2220 and 2230 does not detect any motion of the cameras 2200 and2210, the devices 2220 and 2230 may provide no detection data or provideinformation data representing no motion detection to the combiners 2240and 2250. In one embodiment of the invention, each of the motiondetection devices 2220 and 2230 comprises a typical motion detectionsensor. The motion detection sensor may provide textual or graphicaldetection data to the combiners 2240 and 2250.

[0191] The combiners 2240 and 2250 combine the image data and the motiondetection data, and provide the combined data 2260 and 2270 to thetransmitters 2280 and 2290, respectively. If the combiners 2240 and 2250receive information data representing no motion detection from themotion detection devices 2220 and 2230, or if the combiners 2240 and2250 do not receive any motion data, each combiner (2240, 2250) providesonly the image data to the transmitters 2280 and 2290 without motiondetection data. In one embodiment of the invention, each of thecombiners 2240 and 2250 comprises a typical multiplexer. Each of thetransmitters 2280 and 2290 transmits the combined data 2260 and 2270 tothe display site through a communication network (not shown).

[0192]FIG. 24 illustrates a display system corresponding to the camerasystem shown in FIG. 23. The display system is directed to providecamera motion to a viewer. The camera system comprises a pair ofreceivers 2300 and 2310, data separators 2320 and 2330, image processors2340 and 2360, microcomputers 2350 and 2370, on screen data (OSD)circuits 2390 and 2410, combiners 2380 and 2400, display drivers 2420and 2430, and display screens 2440 and 2450.

[0193] Each of the receivers 2300 and 2310 receives the combined datatransmitted from the camera system, and provides the received data tothe data separators 2320 and 2330, respectively. Each of the dataseparators 2320 and 2330 separates the image data and the motiondetection data from the received data. The image data are provided tothe image processors 2340 and 2360. The motion detection data areprovided to the microcomputers 2350 and 2370. The image processors 2340and 2360 perform typical image data processing for the image data, andprovide the processed data to the combiners 2380 and 2400, respectively.

[0194] Each of the microcomputers 2350 and 2370 determines camera motioninformation from the motion detection data. In one embodiment of theinvention, each microcomputer (2350, 2370) determines camera motioninformation for at least four directions, e.g., upper, lower, left,right. The microcomputers 2350 and 2370 provide the determined cameramotion information to the OSD circuits 2390 and 2410, respectively. Eachof the OSD circuits 2390 and 2410 produces OSD data representing cameramotion based on the determined motion information. In one embodiment ofthe invention, the OSD data comprise arrow indications 2442-2448 showingthe motions of the cameras 2200 and 2210. The arrows 2442 and 2448 meanthat the camera has moved to the upper and lower directions,respectively. The arrows 2444 and 2446 mean that the camera has moved tothe left and right directions, respectively.

[0195] The combiners 2380 and 2400 combine the processed image data andthe OSD data, and provide the combined image to the display drivers 2420and 2430. Each of the display drivers 2420 and 2430 displays thecombined image in each of the display screens 2440 and 2450.

[0196] Referring to FIG. 25, the operation of the camera and displaysystems shown in FIGS. 23 and 24 will be described. Each of thestereoscopic cameras 2200 and 2210 images an object (2460). The pair ofthe motion detection devices 2220 and 2230 detect the motions of thecameras 2200 and 2210, respectively (2470). The photographed image dataand the motion detection data are combined in each of the combiners 2240and 2250 (2480). The combined data 2260 and 2270 are transmitted to thedisplay site through a communication network (2490). Other embodimentsmay not have the combining and separation of data as shown in thediagrams.

[0197] The transmitted data from the camera system are provided to thedata separators 2320 and 2330 via the receivers 2300 and 2310 (2500).The image data and the motion detection data are separated in the dataseparators 2320 and 2330 (2510). The image data are provided to theimage processors 2340 and 2360, and each of the processors 2340 and 2360processes the image data (2520). The motion detection data are providedto the microcomputers 2350 and 2370, and each of the microcomputers 2350and 2370 determines motion information from the motion detection data(2520).

[0198] OSD data corresponding to motion information are generated basedon the determined motion information in the OSD circuits 2390 and 2410(2530). The processed image data and the OSD data are combined togetherin the combiners 2380 and 2400 (2540). The combined data are displayedin the display screens 2440 and 2450 (2550). When the OSD data aredisplayed on the display screens 2440 and 2450, this means that at leastone of the cameras 2200 and 2210 has moved. Thus, the image also movesin the direction in which the cameras 2200 and 2210 have moved. This isfor guiding a viewer's eye lenses to track the motion of the cameras2200 and 2210. In one embodiment of the invention, the arrows 2442-2448are displayed right before the image is moved by the movement of thecameras so that a viewer can expect the movement of the images inadvance.

[0199] In another embodiment of the invention, the display system mayallow the viewer to know the movement of the cameras 2200 and 2210 byproviding a voice message that represents the movement of the cameras.By way of example, the voice message may be “the stereoscopic camerashave moved in the upper direction” or “the cameras have moved in theright direction.” In this embodiment of the invention, the OSD circuits2390 and 2410 may be omitted. In another embodiment of the invention,both of the OSD data and voice message representing the movement of thecameras may be provided to the viewer.

[0200] In one embodiment of the invention, the camera and displaysystems shown in FIGS. 23 and 24 comprise the functions in which theimage is displayed such that the photographing ratio (A:B:C) equals thescreen ratio (A:B:C) as discussed with regard to FIGS. 7-11. In anotherembodiment of the invention, the systems may comprise the function thatdisplays stereoscopic images such that the distance between the centerpoints of the stereoscopic images are substantially the same as thedistance between the center points of a viewer's eyes as discussed withregard to FIGS. 15-22.

[0201] Another aspect of the invention provides a 3D display system thatcontrols the movement of the cameras according to a viewer's eye lensmovement. Before describing the aspect of the invention, therelationship between a viewer's eyes and a set of stereoscopic cameraswill be described by referring to FIGS. 26-28.

[0202]FIG. 26A is a conceptual drawing that illustrates parameters forstereoscopic cameras. Each of the cameras 30 and 32 comprises objectlenses 34 and 36, respectively. The camera parameters comprise C_(2L),C_(2R), C_(3L), C_(3R), S_(CL), S_(CR), V_(c)and W_(C). C_(2L) andC_(2R) represent the center points of the object lenses 34 and 36,respectively. C_(3L) and C_(3R) represent rotation axes of the cameras30 and 32, respectively. S_(CL) represents the line connecting C_(2L)and C_(3L). S_(CR) represents the line connecting C_(2R) and C_(3R).V_(c) represents the distance between C_(2L) and C_(2R). W_(c)represents the distance between C_(3L) and C_(3R).

[0203] The rotation axes C_(3L) and C_(3R) do not move and are the axesaround which the cameras 30 and 32 rotate. The rotation axes C_(3L) andC_(3R) allow the cameras 30 and 32 to rotate by behaving like a carwindshield wiper, respectively, as shown in FIGS. 27B-27E. FIG. 27Aillustrates a default position of the cameras 30 and 32. FIGS. 27B-27Dillustrate the horizontal movements of the cameras 30 and 32. FIG. 27Eillustrates the vertical movements of the cameras 30 and 32. In oneembodiment of the invention, while they are moving and after they moveas shown in FIGS. 27B-27E, each of the cameras 30 and 32 issubstantially parallel to each other. FIG. 27F is a front view of one ofthe stereoscopic cameras and exemplifies the movements of the camera ineight directions. The diagonal movements 46 a-46 d may be performed bythe combination of the horizontal and vertical movements. For example,the movement “46 a” is made by moving the camera to the left and upperdirections.

[0204]FIG. 26B is a conceptual drawing that illustrates parameters for aviewer's eyes. Each of the eyes 38 and 40 comprises eye lenses 42 and44, respectively. Each of the eye lenses is located substantially in theoutside surface of the eyes. This means that the distance between eachcenter point of the eyes and each eye lens is substantially the same asthe radius of the eye. The eye lens moves along with the rotation of theeye. The eye parameters comprise A_(2L), A_(2R), A_(3L), A_(3R), S_(AL)_(, S) _(AR), V_(a) and W_(a). A_(2L) and A_(2R) represent the centerpoints of the eye lenses 42 and 44, respectively. Each of the eye lenses42 and 44 performs substantially the same function as the object lenses34 and 36 of the stereoscopic cameras 30 and 32 in terms of receiving animage. Thus, the eye parameters A_(2L) and A_(2R) may correspond to thecamera parameters C_(2L) and C_(2R).

[0205] A_(3L) and A_(3R) represent rotation axes of the eyes 38 and 40,respectively. The rotation axes A_(3L) and A_(3R) are the axes aroundwhich the eyes 38 and 40 rotate. The rotation axes A_(3L) and A_(3R)allow the eyes 38 and 40 to rotate as shown in FIGS. 28B-28D. As therotation axes C_(3L) and C_(3R) of the stereoscopic cameras 30 and 32 donot move while the cameras 30 and 32 are rotating, so the rotation axesA_(3L) and A_(3R) of a viewer's eyes 38 and 40 do not move while theeyes 38 and 40 are rotating. Thus, the eye parameters A_(3L) and A_(3R)may correspond to the camera parameters C_(3L) and C_(3R).

[0206] S_(AL) represents the line connecting A_(2L) and A_(3L). S_(AR)represents the line connecting A_(2R) and A_(3R). As shown in FIGS. 26Aand 26B, the eye parameters S_(AL) and S_(AR) may correspond to thecamera parameters S_(CL) and S_(CR), respectively. V_(a) represents thedistance between A_(2L) and A_(2R). W_(a) represents the distancebetween A_(3L) and A_(3R). Similarly, the eye parameters V_(a) and W_(a)may correspond to the camera parameters V_(c) and W_(c), respectively.

[0207] Referring to FIGS. 28A-28C, it can be seen that when thedirections of the eyes 38 and 40 change, only the directions of S_(AL)and S_(AR) change while the rotations axes A_(3L) and A_(3R) are fixed.This means that W_(a) is constant while the lines S_(AL) and S_(AR)change. Thus, in order to control the movements of the cameras 30 and 32based on the movements of the eyes 38 and 40, the directions of thecamera lines S_(CL) and S_(CR), need to be controlled based on those ofeye lines S_(AL) and S_(AR) while the distance W_(c) is constant.

[0208]FIG. 28A illustrates an example of the eye configuration in whicha viewer sees an object at least “10,000 m” distant from him or her.This example corresponds to the camera configuration in which the focallength of the cameras are infinity. As discussed before, when a viewersees an object farther than, for example, “10,000 m,” the distance(V_(a)) between the center points A_(2L) and A_(2R) of the eye lenses 42and 44 is substantially the same as the distance (W_(a)) between thecenter points A_(3L) and A_(3R) of the eyes 38 and 40.

[0209] When a viewer sees an object that is located in front of him orher and is closer than, for example, “10 m,” the viewer's left eyerotates in a clockwise direction and right eye rotates in a counterclockwise direction as shown in FIG. 28B. Consequently, the distanceV_(a) becomes shorter than the distance W_(a). If a viewer sees anobject that is located in a slightly right front side of him or her,each of the eyes rotates in a clockwise direction as shown in FIG. 28C.In this situation, the distance V_(a) may be less than the distanceW_(a). FIG. 28D exemplifies the movements of the eyes in eightdirections.

METHOD AND SYSTEM FOR CONTROLLING THE MOTION OF STEREOSCOPIC CAMERASBASED ON A VIEWER'S EYE LENS MOTION

[0210]FIG. 29 illustrates a 3D display system for controlling a set ofstereoscopic cameras according to another aspect of the invention. Thesystem comprises a camera site and a display site. The display site isdirected to transmit eye lens motion data to the camera site. The camerasite is directed to control the set of stereoscopic cameras 30 and 32based on the eye lens motion data.

[0211] The display site comprises an eye lens motion detecting device3000, a transmitter 3010, a pair of display devices 2980 and 2990, apair of receivers 2960 and 2970, and a V shaped mirror 2985. When thecamera site transmits stereoscopic images through a pair of transmitters2900 and 2930 to the display site, the display site receives the imagesand displays through the display devices 2980 and 2990. A viewer seesstereoscopic images through the V shaped mirror that reflects thedisplayed image to the viewer. While the viewer is watching the images,the viewer's eye lenses may move in directions, e.g., latitudinal (upperor lower) and longitudinal (clockwise or counterclockwise) directions.Once again, another display device such as a HMD, or a projectiondisplay device as discussed above, may be used.

[0212] The eye lens motion detecting device 3000 detects motions of eachof a viewer's eye lenses while a viewer is watching 3D images throughthe V shaped mirror. The motions may comprise current locations of theeye lenses. The detecting device 3000 is substantially the same as thedevice 2100 shown in FIG. 21A. The detecting device 3000 may convert themovements of the eye lenses to data that a microcomputer 2940 of thecamera site can recognize, and provide the converted data to thetransmitter 3010. In one embodiment of the invention, the detection datamay comprise a pair of (x,y) values for each of the eye lenses.

[0213] The transmitter 3010 transmits the eye lens motion data to thecamera site through a communication network 3015. The detection data maycomprise identification data that identify each of the left and righteye lenses in the camera site. In one embodiment of the invention, thedisplay site may comprise a pair of transmitters each transmitting leftand right eye lens motion data to the camera site. In one embodiment ofthe invention, before transmitting the motion data, data modificationsuch as encoding and/or modulation adapted for transmitting may beperformed.

[0214] The camera site comprises a set of stereoscopic cameras 30 and32, a receiver 2950, a microcomputer 2940, a pair of camera controllers2910 and 2920, the pair of transmitters 2900 and 2930. The receiver 2950receives the eye lens motion data from the display site, and providesthe data to the microcomputer 2940. The microcomputer 2940 determineseach of the eye lens motion data from the received data, and providesthe left and right eye lens motion data to the camera controllers 2910and 2920, respectively. In one embodiment of the invention, the camerasite may comprise a pair of receivers each of which receives left andright eye lens motion data from the display site, respectively. In thatsituation, each receiver provides each eye lens detection data tocorresponding camera controllers 2910 and 2920, respectively, and themicrocomputer 2940 may be omitted.

[0215] The camera controllers 2910 and 2920 control each of the cameras30 and 32 based on the received eye lens motion data. That is, thecamera controllers 2910 and 2920 control movement of each of the cameras30 and 32 in substantially the same directions as each of the eye lenses42 and 44 moves. Referring to FIG. 30, the camera controllers 2910 and2920 comprise servo controllers 3140 and 3190, horizontal motors 3120and 3160, and vertical motors 3130 and 3180, respectively. Each of theservo controllers 3140 and 3190 controls the horizontal and verticalmotors (3120, 3160, 3130, 3180) based on the received eye lens motiondata. Each of the horizontal motors 3120 and 3160, respectively movesthe cameras 30 and 32 in the horizontal directions. Each of the verticalmotors 3130 and 3180, respectively moves the cameras 30 and 32 in thevertical directions.

[0216]FIG. 31 illustrates a flow chart showing the operation of thecamera controllers 2910 and 2920 according to one aspect of theinvention. FIG. 32A illustrates a table for controlling horizontal andvertical motors. FIG. 32B illustrates a conceptual drawing that explainsmotion of the camera. Referring to FIGS. 31 and 32, the operation of thecamera controllers 2910 and 2920 will be described. Since the operationof the camera controllers 2910 and 2920 are substantially the same, onlythe operation of the camera controller 2910 will be described. The servocontroller 3140 initializes camera adjusting values (3200). In oneembodiment of the invention, the initialization of the camera adjustingvalues may comprise setting a default value, for example, “(x,y)=(0,0)”which means no movement. These values correspond to the eye lens motiondata detected in a situation where a viewer sees the front directionwithout moving their eye lenses. In one embodiment of the invention, theinitialization may comprise setting the relationship between theadjusting values and the actual movement amount of the camera 30 asshown in FIG. 32.

[0217] The eye lens motion data are provided to the servo controller3140 (3210). In one embodiment of the invention, the eye lens motiondata comprise, (x,y) coordinate values, where x and y represent thehorizontal and vertical motions of each of the eye lenses, respectively.

[0218] The servo controller 3140 determines camera adjusting values (X,Y) based on the provided eye lens motion data. It is determined whetherX equals “0” (3230). If X is “0,” the servo controller 3140 does notmove the horizontal motor 3120 (3290). If X is not “0,” it is determinedwhether X is greater than “0” (3240). If X is greater than “0,” theservo controller 3140 operates the horizontal motor 3120 to move thecamera 30 in the right direction (3270). As exemplified in FIG. 32A, ifthe value X is, for example, “1,” the movement amount is “2°,” and thedirection is clockwise (θ₃ direction). If the value X is, for example,“2,” the movement is “4°” in a clockwise direction.

[0219] If X is not greater than “0,” meaning this means that X is lessthan “0,” the servo controller 3140 operates the horizontal motor 3120so as to move the camera 30 in a counterclockwise (θ₁) direction (3260).Referring to FIG. 32, if the value X is, for example, “−1,” the movementamount is “2°,” and the direction is counterclockwise. If the value xis, for example, “−3,” the movement is “6°” in a counterclockwise (θ₁)direction.

[0220] Similarly, it is determined whether Y equals “0” (3300). If Y is“0,” the servo controller 3140 does not move the vertical motor 3130(3290). If Y is not “0,” it is determined whether Y is greater than “0”(3310). If Y is greater than “0,” the servo controller 3140 operates thevertical motor 3130 to move the camera 30 to +latitudinal (upper: θ₂)direction (3320). If the value Y is, for example, “2,” the movement is“4°” in the upper direction.

[0221] If Y is not greater than “0,” the servo controller 3140 operatesthe vertical motor 3130 so as to move the camera 30 in the lowerdirection (3330). If the value Y is, for example, “−3,” the movementamount is “6°,” and the direction is in a −latitudinal (lower: θ₄)direction.

[0222] Now, the entire operation of the system shown in FIG. 29 will bedescribed with reference to FIG. 33. The eye lens motion detectiondevice 3000 is provided to the display site of the system (3020). Aviewer's eye lens motion is detected by the eye lens motion detectiondevice 3000 while the viewer is watching stereoscopic images (3030). Theeye lens motion data are transmitted to the camera site through thetransmitter 3010 and the communication network 3015 (3040). As discussedabove, either one transmitter or a pair of transmitters may be used.

[0223] The receiver 2950 of the camera site receives the eye lens motiondata from the display site (3050). The camera adjusting values aredetermined based on the eye lens motion data (3060). The stereoscopiccameras 30 and 32 are controlled by the determined camera adjustingvalues (3070). In this way, the stereoscopic cameras 30 and 32 arecontrolled such that the cameras keep track of the eye lens motion. Interms of the viewer, he or she notices that as soon as his or her eyelenses are moved to a certain direction, stereoscopic images are alsomoved in the direction to which the eye lenses has moved.

[0224]FIG. 34 illustrates a stereoscopic camera controller system usedfor a 3D display system according to another aspect of the invention.For convenience, the display site is not shown. This aspect of theinvention selects a pair of stereoscopic cameras corresponding tomovement amount of the eye lenses among plural sets of stereoscopiccameras instead of controlling the movement of the pair of stereoscopiccameras.

[0225] The system comprises a microcomputer 3430, a memory 3440, cameraselectors 3420 and 3425, and plural sets of stereoscopic cameras 30 aand 32 a, 30 b and 32 b, and 30 c and 32 c. The memory 3440 stores atable as shown in FIG. 35. The table shows relationship between cameraadjusting values and selected cameras. The camera adjusting value“(0,0)” corresponds to, for example, a set of cameras C33 as shown inFIGS. 35 and 36B. The camera adjusting value “(1,0)” corresponds to aset of cameras C34 as shown in FIGS. 35 and 36B. The camera adjustingvalue “(2,2)” corresponds to the C15 camera set as shown in the Figures.In one embodiment of the invention, another set of stereoscopic camerasis selected from the sets of cameras such as one of the C34 camera setand one of the C32 camera set.

[0226]FIG. 36A is a top view of the plural sets of stereoscopic cameras.In one embodiment of the invention, the contour line that is made byconnecting all of the object lenses of the plural sets of stereoscopiccameras is similar to the contour line of a viewer's eyes which isexposed to the outside.

[0227] The microcomputer 3430 determines camera adjusting values basedon the received eye lens motion data. The microcomputer 3430 alsodetermines first and second camera selection signals based on the tablestored in the memory 3440. The first selection signal is determinedbased on the movement of a viewer's left eye lens, and used forcontrolling the camera selector 3420. The second selection signal isdetermined based on the movement of a viewer's right eye lens, and usedfor controlling the camera selector 3425. The microcomputer 3430provides each of the selection signals to the camera selectors 3420 and3425, respectively.

[0228] The camera selectors 3420 and 3425 select the respective camerabased on the selection signal. In one embodiment of the invention, abase set of cameras (e.g., C33) shown in FIG. 36B, image an object andtransmit the image to the display site through the transmitters 2900 and2930, respectively. In this embodiment of the invention, if the cameraselectors 3420 and 3425 select another set of cameras, the selected setof cameras image the object and transmit the image to the display sitethrough the transmitters 2900 and 2930. In one embodiment of theinvention, all of the cameras are turned on and a first set of camerasare connected to the transmitters 2900 and 2930, respectively. In thisembodiment of the invention, when a second set of cameras are selected,the first set of cameras are disconnected from the transmitters 2900 and2930, and the second set of cameras are connected to the transmitters2900 and 2930, respectively. In another embodiment of the invention,only a selected set of cameras are turned on and the non-selected set ofcameras remain turned off. In one embodiment of the invention, each ofthe camera selectors 3420 and 3425 comprises a switch that performsswitching between the plural sets of stereoscopic cameras 30 a and 32 a,30 b and 32 b, and 30 c and 32 c and the transmitters 2900 and 2925,respectively.

[0229] Referring to FIG. 37, the operation of the system shown in FIG.34 will be described. A base set of cameras (e.g., C33) of FIG. 36,image an object (3710). Eye lens motion data are received from thedisplay site (3720). Camera adjusting values are determined based on thereceived eye lens motion data (3730). The camera adjusting values areexemplified in the table of FIG. 35. Camera selection signals aredetermined based on the determined camera adjusting values (3740), forexample, using the relationship of the table of FIG. 35. It isdetermined whether a new set of cameras have been selected (3750). If nonew set of cameras are selected, the image output from the base camerasis transmitted to the display site (3780). If a new set of cameras(e.g., C35) is selected, the base cameras (C33) are disconnected fromthe transmitter 2900 and the new cameras (C35) are connected to thetransmitters 2900 and 2930 (3760). The selected cameras (C35) image theobject (3770), and the image output from the selected cameras istransmitted to the display site (3790).

[0230] Regarding the embodiments described with regard to FIGS. 29-37,the camera control may be used in remote control technology such as aremote surgery, remote control of a vehicle, an airplane, or aircraft,fighter, or remote control of construction, investigation or automaticassembly equipments.

METHOD AND SYSTEM OF STEREOSCOPIC IMAGE DISPLAY FOR GUIDING A VIEWER'SEYE LENS MOTION USING A THREE-DIMENSIONAL MOUSE

[0231]FIG. 38 illustrates a 3D display system according to anotheraspect of the invention. The 3D display system is directed to guide aviewer's eye lens motion using a three-dimensional input device. Thesystem is also directed to adjust displayed images using the 3D inputdevice such that the longitudinal and latitudinal locations of thecenter points of a viewer's eye lenses are substantially the same asthose of the center points of the displayed images. In one embodiment ofthe invention, the 3D input device comprises a 3D mouse (will bedescribed later).

[0232] The system comprises a set of stereoscopic cameras 30 and 32, apair of transmitters 2900 and 2930, a set of display devices 3900 and3910, a 3D mouse 3920, and an input device 3990. The stereoscopiccameras 30 and 32, a pair of transmitters 2900 and 2930, and a pair ofreceivers 2960 and 2970 are the same as those shown in FIG. 29. Thedisplay devices 3900 and 3910 display stereoscopic image that has beentransmitted from the camera site. Also, the devices 3900 and 3910display the pair of 3D mouse cursors that guide a viewer's eye lensmovement.

[0233] In one embodiment of the invention, the input of the 3D mouse isprovided to both the display devices 3900 and 3910 as shown in FIG. 38.In this embodiment of the invention, the pair of 3D mouse cursors aredisplayed and moved by the movement of the 3D mouse 3920.

[0234] In one embodiment of the invention, the shape of the 3D mousecursor comprises a square, an arrow, a cross, a square with a crosstherein as shown in FIGS. 40A-40H, a reticle, or a crosshair. In oneembodiment of the invention, a pair of cross square mouse cursors 400and 420 as shown in FIG. 40 will be used for the convenience. In oneembodiment of the invention, when a viewer adjusts a distance value(will be described in more detail referring to FIGS. 39 and 40) for thedisplayed images, the distance (M_(d)) between the 3D mouse cursors 400and 420 is adjusted. Also, in this embodiment of the invention, the sizeof the 3D mouse cursors may be adjusted. In this embodiment of theinvention, the viewer adjusts the distance value, for example, byturning a scroll button of the 3D mouse. For example, by turning thescroll button backward (towards the user), the viewer can set a distancevalue from a larger value to a smaller one (10,000 m −>100 m −>5 m −>1 m−>0.5 m −>5 cm). Also, by turning the scroll button forward (oppositedirection of the backward direction), the viewer may set a distancevalue from a smaller value to a larger one (5 cm −>0.5 m −>1 m −>5 m−>100 m −>10,000 m). Hereinafter the distance value 10,000 m will veryoften be referred to as an infinity value or infinity.

[0235]FIG. 39 illustrates one example of a 3D display image. The imagecomprises a mountain image portion 3810, a tree image portion 3820, ahouse image portion 3830 and a person portion image 3840. It is assumedthat the mountain image 3810, the tree image 3820, the house image 3830,the person image 3840 are photographed in distances “about 10,000 m,”“about 100 m,” “about 5 m,” and “about 1 m,” respectively, spaced fromthe set of stereoscopic cameras 30 and 32.

[0236] When a viewer wants to see the mountain image 3810 shown in FIG.39, he or she may set the distance value as a value greater than “10,000m.” In this situation, the mouse cursor distance M_(d) has M_(d0) valuewhich is the same as the W_(a) (V_(amax)) values as shown in FIG. 40A.As discussed above, when the viewer sees an infinity object, V_(a) hasthe maximum value (V_(amax)). Also, the viewer's sight lines L_(s1) andL_(s2), each of which is an extended line of each of S_(AL) and S_(AR)(each connecting A₂ and A₃), are substantially parallel to each other asshown in FIGS. 40A and 40B. This means that if the viewer sees thedisplayed images with their eye lenses spaced as much as W_(a) as shownin FIGS. 40A and 40B, the viewer feels a sense of distance as if theysee an object that is “d₀ (10,000 m)” distant. This is because a humanbeing's eyes are spaced apart from each other about 60-80 mm and a senseof 3 dimension is felt by the synthesized images of each eye in thebrain. Thus, when the viewer sees the two mouse cursors that are spacedas much as M_(d) =W_(a), they perceive a single (three-dimensional)mouse cursor that is located between the two mouse cursors (400, 420) atan infinity distance.

[0237] When the viewer sets the distance value (d₁) to, for example,“100 m,” and sees the tree image 3820, In this situation, M_(d) hasM_(d1) value which is less than M_(d0) as shown in FIGS. 40C and 40D.Also, the viewer's sight lines L_(s1) and L_(s2) are not parallel anymore. Thus, when the two sight lines are extended, they are converged inan imaginary point “M” as shown in FIG. 40D. the point “O” representsthe middle point between the center points of each eye. Similarly, ifthe viewer sees the displayed images with their eye lenses spaced asmuch as M_(d1) as shown in FIGS. 40C and 40D, the viewer feels a senseof distance as if they see an object that is “d₁ (100 m)” distant. Thedistance between M and O is not physical length but imaginary length.However, since the viewer feels a sense of the distance, as far as theviewer's eye lens distance or directions are concerned, the distancebetween M and O can be regarded as the actual distance between theviewer's eyes and an actual object. That is, when the viewer sees thetwo mouse cursors 400 and 420 that are spaced as much as M_(d1), theyperceive a single (three-dimensional) mouse cursor that is located inthe M point, at a 100 m distance.

[0238] When the viewer sets a smaller distance value (d₂) to, forexample, “5 m” and sees the house image 3830, M_(d) has M_(d2) valuewhich is less than M_(d1) as shown in FIGS. 40E and 40F. Also, when thetwo sight lines are extended in the screen, they are converged in animaginary point “M” as shown in FIG. 40F. Similarly, in this situationwhen the viewer sees the house image 3830, the viewer feels a sense ofdistance as if he or she sees an object that is “d₂ (5 m)” away. Thus,when the viewer sees the two mouse cursors 400 and 420 that are spacedas much as M_(d2), they perceive a single (three-dimensional) mousecursor that is located in the M point, at a 5 m distance.

[0239] When the viewer sets a distance value (d₃) to the distancebetween the viewer and the screen, as exemplified as “50 cm,” the mousecursors 400 and 420 overlap with each other as shown in FIG. 40G. Thatis, when the distance value is the same as the actual distance betweenthe point “O” and the center points of the screen as shown in FIG. 40G,the mouse cursors overlap with each other.

[0240] As seen in FIGS. 40A-40G, even though a pair of the 3D mousecursors 400 and 420 are displayed in each of the display devices 3900and 3910, the viewer sees one three-dimensional 3D mouse cursor forwhich he or she feels a sense of distance.

[0241] When the viewer sets the distance value to a value (d₄) less than“d₃,” the viewer's sight lines are converged in front of the screen andcrossed to each other as shown in FIG. 40H. In this situation, theviewer may see two mouse cursors 400 and 420 because the viewer's sightlines are converged in front of the screen.

[0242] As shown in FIGS. 40A-40H, the M_(d) value is determinedaccording to the distance value that is set by the viewer.

[0243]FIG. 41 illustrates an exemplary block diagram of the displaydevices as shown in FIG. 38. Since each of the display devices 3900 and3910 performs substantially the same functions, only one display device3900 is illustrated in FIG. 41.

[0244] The display device 3900 comprises a display screen 3930, adisplay driver 3940, a microcomputer 3950, a memory 3960 and Interfaces3970 and 3980. The display device 3900 adjusts the distance (M_(d))between a pair of 3D mouse cursors 400 and 420 according to the distancevalue set as shown in FIGS. 40A-40H. The display device 3900 moves thecenter points of the displayed images based on the 3D mouse cursormovement. In one embodiment of the invention, the display device 3900moves the displayed images such that the longitudinal and latitudinallocations of the center points of a viewer's eye lenses aresubstantially the same as those of the center points of the displayedimages.

[0245] The 3D mouse 3920 detects its movement amount. The detectedmovement amount is provided to the microcomputer 3950 via the interface3970. The distance value that the viewer sets is provided to themicrocomputer 3950 via the 3D mouse 3920 and the interface 3970. In oneembodiment of the invention, the interface 3970 comprises a mousecontroller. In another embodiment of the invention, the distance valuemay be provided to the microcomputer 3950 via the input device 3990 andthe interface 3980.

[0246] The input device 3990 provides properties of the 3D mouse such asminimum detection amount (A_(m)), movement sensitivity (B_(m)), and themouse cursor size (C_(m)), the viewer-screen distance (d), and viewer'seye data such as W_(a) and S_(AL) and S_(AR) to the microcomputer 3950via the interface 3980. The minimum detection amount represents theleast amount of movement which the 3D mouse can detect. That is, whenthe 3D mouse moves only more than the minimum detection amount, themovement of the 3D mouse can be detected. In one embodiment of theinvention, the minimum detection amount is set when the 3D mouse ismanufactured. The movement sensitivity represents how sensitive themouse cursors move based on the movement of the 3D mouse. This meansthat the scroll button of the 3D mouse has different movementsensitivity, i.e., being either more sensitive or less sensitive,according to the distance value. For example, if the distance value isgreater than 1,000 m, a “1 mm turn” of the scroll button may increase ordecrease the distance by 2,000 m distance. If the distance value isbetween 100 m and 1,000 m, a “1 mm turn” of the scroll button mayincrease or decrease distance by 100 m . Similarly, if the distancevalue is less than 1 m, a “1 mm turn” of the scroll button may increaseor decrease the distance by 10 cm.

[0247] In one embodiment of the invention, the mouse cursor size mayalso be adjusted. The distance (d) represents the distance between themiddle point of the viewer's eyes and the screen as exemplified in FIG.43A. In one embodiment of the invention, the screen comprises a V shapedmirror, a HMD screen, a projection screen, and a display device screenas shown in FIG. 1B.

[0248] Also, the input device 3990 provides display device properties tothe microcomputer 3950 through the interface 3980. In one aspect of theinvention, the display device properties comprise the display deviceresolution and screen size of the display device 3900. The resolutionrepresents the number of horizontal and vertical pixels of the device3900. For example, if the resolution of the display device 3900 is640×480, the number of the horizontal pixels is 640, and the number ofthe vertical pixels is 480. The size may comprise horizontal andvertical lengths of the display device 3900. With the resolution andscreen size of the display device 3900, the length of one pixel can beobtained as, for example, “1 mm” per 10 pixels.

[0249] In one embodiment of the invention, the input device 3990comprises a keyboard, a remote controller, and a pointing input device,etc. In one embodiment of the invention, the interface 3980 comprisesthe input device controller. In one embodiment of the invention, theproperties of the 3D mouse are stored in the memory 3960. In oneembodiment of the invention, the viewer's eye data are detected using adetection device for eye lens movement or provided to the display device3900 by the viewer.

[0250] The microcomputer 3950 determines the mouse cursor distance(M_(d)) based on the distance value set by the viewer. A table (notshown) showing the relationship between the distance value and the M_(d)value as shown in FIGS. 40A-40H according to a viewer's eye data may bestored in the memory 3960. The microcomputer 3950 determines the cursordistance (M_(d)) by referring to the table, and provides the determineddistance value to the display driver 3940. The display driver 3940displays the pair of the mouse cursors 400 and 420 based on thedetermined M_(d) value in the display screen 3930. The microcomputer3950 also determines new locations of the mouse cursors 400 and 420, andcalculates a movement amount for the center points of the display imagesbased on the locations of the mouse cursors 400 and 420. The memory 4730may also store data that may be needed to calculate the movement amountfor the center points of the display images.

[0251] Referring to FIG. 42, the operation of the display devices 3900and 3910 will be described. 3D mouse properties are set in each of thedisplay devices 3900 and 3910 (4200). As discussed above, the 3D mouseproperties comprise a minimum detection amount (A_(m)), a movementsensitivity (B_(m)), and the mouse cursor size (C_(m)). Also, the 3Dmouse properties may be provided by the viewer or stored in the memory3960.

[0252] Display device properties are provided to the display devices3900 and 3910 (4205). In one embodiment of the invention, the displaydevice properties may be stored in the memory 3960.

[0253] The viewer's eye data are provided to the display devices 3900and 3910 (4210). As discussed above, the viewer's eye data may beautomatically detected by a detection device or provided to the displaydevices 3900 and 3910 by the viewer. In one embodiment of the invention,the viewer's eye data comprise the distance (W_(a)) between the centerpoints of the eyes, and the S_(A) (S_(AL) and S_(AR)) value which is thedistance between the eye lens center point (A₂) and the eye center point(A₃).

[0254] A viewer-screen distance (d) is provided to each of the displaydevices 3900 and 3910 via, for example, the input device 3990 (4220).

[0255] The mouse cursor location and distance value are initialized(4230). In one embodiment of the invention, the initialization isperformed in an infinity distance value. In this situation, left andright mouse cursors are located at (−W_(a)/2, 0, 0) and (W_(a)/2, 0, 0),respectively, where the origin of the coordinate system is 0 (0, 0, 0)point as shown in FIG. 43A. Also, the locations of the center points ofeach displayed image are (−W_(a)/2, 0, 0) and (W_(a)/2, 0, 0),respectively.

[0256] 3D image and 3D mouse cursors are displayed in each of thedisplay devices 3900 and 3910 (4240). In one embodiment of theinvention, 3D mouse cursors 400 and 420 are displayed on each of the 3Dimages. Since the mouse cursor location has been initialized, theadjusted mouse cursors 400 and 420 are displayed on the images.

[0257] It is determined whether initialized distance value has beenchanged to another value (4250). When the viewer may want to setdifferent distance value from the initialized distance value, he or shemay provide the distance value to the display devices 3900 and 3910.

[0258] If the initialized distance value has been changed, 3D mousecursor distance (M_(d)) is adjusted and the 3D mouse cursor location isreinitialized based on the changed distance value (4260). For example,in case that the initial location is (0, 0, 10,000 m), if anotherdistance value (e.g., 100 m) as shown in FIG. 40C is provided, the mousecursor distance (M_(d)) is changed from M_(d0) to M_(d1). However, the xand y values of the point M do not change, even though the z value ofthe M point is changed from 10,000 m to 100 m.

[0259] If the initialized distance value has not been changed, it isdetermined whether 3D mouse movement has been detected (4270).

[0260] If the 3D mouse movement has been detected, a new location of the3D mouse cursors 400 and 420 is determined (4280). In one embodiment ofthe invention, the new location of the mouse cursors is determined asfollows. First, the number of pixels on which the mouse cursors havemoved in the x-direction is determined. For example, left directionmovement may have “−x” value and right direction movement may have “+x”value. The same applies to “y” direction, i.e., “−y” value for lowerdirection movement and “+y” value for upper direction movement. The “z”direction movement is determined by the distance value.

[0261] The locations of the center points of the display images to beadjusted are calculated based on the new location of the 3D mousecursors 400 and 420 (4290). In one embodiment of the invention, thelocations of the center points of the display images are obtained fromthe location values of each of the eye lenses, respectively. In thisembodiment of the invention, the location values of the eye lenses areobtained using Equations VII and VIII as described below. Referring toFIG. 43, a method of obtaining the locations of the eye lenses will bedescribed.

[0262] First, the value for Z_(L) is obtained from Equation VII.$\begin{matrix}{{{Equation}\quad \text{VII:}}{Z_{L} = {\sqrt{\left\lbrack {I_{N} - \left( {- \frac{W_{a}}{2}} \right)} \right\rbrack^{2} + \left\lbrack {J_{N} - 0} \right\rbrack^{2} + \left\lbrack {K_{N} - 0} \right\rbrack^{2}} = \sqrt{\left\lbrack {I_{N} + \left( \frac{W_{a}}{2} \right)} \right\rbrack^{2} + \left\lbrack J_{N} \right\rbrack^{2} + \left\lbrack K_{N} \right\rbrack^{2}}}}} & \quad\end{matrix}$

[0263] In FIG. 43A, M_(N) (I_(N), J_(N), K_(N)) represents the locationof the center point of the two mouse cursors M_(L)(I_(L), J_(L), K_(L))and M_(R) (I_(R), J_(R), K_(R)). Since each of the mouse cursorlocations M_(L) and M_(R) is obtained in 4280, the center point locationM_(N) is obtained. That is, I_(N) and J_(N) are obtained by averaging(I_(L), I_(R)) and (J_(L), J_(R)). K_(N) is determined by the currentdistance value. Z_(L) is the distance between the left eye center point(A_(3L)) and M_(N).

[0264] Second, center point locations [(x1, y1, z1); (x2, y2, z2)] foreach eye lens are obtained from Equation VIII. A_(2L) (x1, y1, z1) isthe center point location of the left eye lens, and A_(2R) (x2, y2, z2)is the center point location of the right eye lens, as shown in FIG.43A. FIG. 43B illustrates a three-dimensional view of a viewer's eye.Referring to FIG. 43B, it can be seen how eye lens center point (A_(2L))is moving along the surface of the eye. Equation  VIII:$\begin{matrix}\begin{matrix}{{x1} = {\left( {- \frac{W_{a}}{2}} \right) + \frac{\left\lbrack {\left( {I_{N} + \frac{W_{a}}{2}} \right) \times S} \right\rbrack}{Z_{L}}}} \\{{y1} = {0 + \frac{\left\lbrack {\left( J_{N} \right) \times S} \right\rbrack}{Z_{L}}}} \\{{z1} = {0 + \frac{\left\lbrack {\left( K_{N} \right) \times S} \right\rbrack}{Z_{L}}}} \\{{x2} = {\left( \frac{W_{a}}{2} \right) - \frac{\left\lbrack {\left( {I_{N} + \frac{W_{a}}{2}} \right) \times S} \right\rbrack}{Z_{L}}}} \\{{y2} = {0 + \frac{\left\lbrack {\left( J_{N} \right) \times S} \right\rbrack}{Z_{L}}}} \\{{z2} = {0 + \frac{\left\lbrack {\left( K_{N} \right) \times S} \right\rbrack}{Z_{L}}}}\end{matrix} & \quad\end{matrix}$

[0265] In one embodiment of the invention, a digital signal processormay be used for calculating the locations of the eye lenses.

[0266] Each of the center points of the displayed images is moved to thelocations (x1, y1) and (x2, y2), respectively as shown in FIG. 44(4300). In one embodiment of the invention, the blank area of the screenafter moving may be filled with a background color, e.g., black, asshown in FIG. 44.

[0267] It is determined whether the 3D mouse movement has been completed(4310). If the 3D mouse movement has not been completed, procedures4280-4300 are performed until the movement is completed. This ensuresthat the displayed images are moved so long as the viewer is moving themouse cursor.

[0268] By using the above calculation method, the distance between twolocations can be measured. Referring to FIG. 43C, M_(N1) is a peak pointof a mountain 42 and M_(N2) is a point of a house 44. It is assumed thatthe location values of M_(N1) and M_(N2) are determined to be (−0.02 m,0.04 m, 100 m) and (0.01 m, 0 m, 10 m), respectively by the abovecalculation method. These determined location values may be stored inthe memory 3960, and the distance between the two locations M_(N1) andM_(N2) is calculated as follows.

Z _(L)={square root}{square root over ([−0.02−0.01]²+[0.04−0]²+[100−10]²)}=90

[0269] In this embodiment, the microcomputer 3950 is programmed tocalculate the distance between two locations, or may comprise a distancemeasure mode. In this situation, when a viewer designates a firstlocation (A: middle point of two mouse cursors 400 and 420), thelocation is determined and stored in the memory 3960. In one embodiment,the location value may be displayed in the display screen 3930 or may beprovided to a viewer via voice signal. This applies to a second location(B). In this way, the values of the first and second locations (A, B)are determined and the distance between the locations (A, B) iscalculated.

METHOD AND SYSTEM FOR CONTROLLING THE MOTION OF STEREOSCOPIC CAMERASUSING A THREE-DIMENSIONAL MOUSE

[0270]FIG. 45 illustrates a 3D display system according to anotheraspect of the invention. The system is directed to control the movementof stereoscopic cameras based on the movement of a viewer's eye lenses.

[0271] The system comprises a camera site and a display site. Thedisplay site comprises a pair of transmitters/receivers 4530 and 4540, aset of display devices 4510 and 4520, and an input device 3990 and a 3Dmouse 3920.

[0272] The input device 3990 and 3D mouse 3920 are substantially thesame as those of the system shown in FIG. 38. Referring to FIG. 46, thedisplay device 4510 comprises interfaces 3970 and 3980, a microcomputer4820, a memory 4830, and an interface 4810. The interfaces 3970 and 3980are substantially the same as those of the display device shown in FIG.41. The microcomputer 4820 determines the current location values of themouse cursors, and calculates the location values of the center pointsof a viewer's eye lenses. The memory 4830 may also store data that maybe needed to calculate the movement amount for the center points of thedisplay images.

[0273] The interface 4810 may modify the location values adapted fortransmission, and provide the modified data to the transmitter 4530. Thetransmitter 4530 transmits the modified location data to the camerasite.

[0274] Referring to FIG. 45, the camera site comprises a set ofstereoscopic cameras 30 and 32, a pair of transmitters 4570 and 4600, apair of servo mechanisms 4580 and 4590, and a pair of receivers 4550 and4560. Each of the receivers 4550 and 4560 receives the location valuestransmitted from the display site, and provides the data to the pair ofthe servo mechanisms, 4580 and 4590, respectively.

[0275] The servo mechanisms 4580 and 4590 control the cameras 30 and 32based on the received location data, respectively. In one embodiment ofthe invention, the servo mechanisms 4580 and 4590 control the cameras 30and 32 such that the longitudinal and latitudinal values of the centerpoints of the object lenses (C_(2L), C_(2R); FIGS. 26 and 27) of thecameras 30 and 32 are substantially the same as those of the centerpoints of the viewer's eye lenses as shown in FIGS. 47A and 47C.

[0276] Referring to FIG. 48, the operation of the system shown in FIG.45 will be described. 3D mouse properties and display device propertiesare set in each of the display devices 4510 and 4520 (4610). The 3Dmouse properties and display device properties are substantially thesame as those explained with regard to FIG. 42. The viewer's eye dataand viewer-screen distance (d) are provided to each of the displaydevices 4510 and 4520 (4620). Again, the viewer's eye data andviewer-screen distance (d) are substantially the same as those explainedwith regard to FIG. 42. 3D mouse cursor location and distance value areinitialized (4630). In one embodiment of the invention, the 3D mousecursor location is initialized to the center points of each of thedisplay device screens, and the distance value is initialized to theinfinity distance value. The 3D image that is received from the camerasite, and 3D mouse cursors (400, 420) are displayed on the displaydevices 4510 and 4520 (4640). In one embodiment of the invention, the 3Dmouse cursor may be displayed on the 3D image. In this situation, theportion of the image under the 3D mouse cursors (400, 420) may not beseen by a viewer.

[0277] It is determined whether 3D mouse movement is detected (4650). Ifmovement is detected, the new location of the 3D mouse cursors isdetermined (4660). The location values of the center points of theviewer's eye lenses are calculated based on the new location of themouse cursors, respectively (4670). The new location and movement of themouse cursors (400, 420) are illustrated in FIG. 47B. The specificmethods for performing the procedures 4650-4670 have been described withregard to FIGS. 42-44.

[0278] The location value data are transmitted to the camera sitethrough each of the transmitter/receivers 4530 and 4540 (4680). Asdiscussed above, the location values are calculated so long as the mousecursor is moving. Thus, the location values may comprise a series ofdata. In one embodiment of the invention, the location values areserially transmitted to the camera site so that the cameras 30 and 32are controlled based on the received order of the location values. Inanother embodiment of the invention, the sequence of the generatedlocation values may be obtained and transmitted to the camera site sothat the cameras 30 and 32 are controlled according to the sequence. Inone embodiment of the invention, the location value data are digitaldata and may be properly modulated for transmission.

[0279] The location value data are received in each of the receivers4550 and 4560 (4690). In one embodiment of the invention, onetransmitter may be used instead of the two transmitters 4530 and 4540.In that situation, one receiver may be used instead of the receivers4550 and 4560.

[0280] Camera adjusting values are determined based on the locationvalues and the stereoscopic cameras 30 and 32 are controlled based onthe camera adjusting values (4700). Each of the servo controllers 4580and 4590 controls the respective camera 30 and 32 such that each of thecenter points of the cameras object lenses keeps track of the movementof the center points of each eye lens (4710). As shown in FIG. 47C, newlocation values A_(2L1) and A_(2R1) corresponding to the new location ofthe 3D mouse cursors are calculated using Equations VIII as discussedabove. Each of the servo controllers 4580 and 4590 controls the cameras30 and 32 such that the center points of each of the camera objectlenses are located in C_(2L1) and C_(2R1) as shown in FIG. 47A. To dothis, the servo controllers 4580 and 4590 may set the location values ofthe center points of the camera object lenses so as to conform to thelocation values of the center points of the eye lenses. In oneembodiment of the invention, the servo controllers 4580 and 4590comprise a horizontal motor and a vertical motor that move each camerato the horizontal direction (x-direction) and the vertical direction(y-direction), respectively. In one embodiment of the invention, onlyone servo controller may be used for controlling movements of both ofthe cameras 30 and 32 instead of the pair of the servo controllers 4580and 4590.

[0281] While each of the servo controllers 4580 and 4590 is controllingthe stereoscopic cameras 30 and 32, the cameras 30 and 32 arephotographing an object. The photographed image is transmitted to thedisplay site and displayed in each of the display devices 4510 and 4520(4720, 4730).

[0282] Regarding the embodiments described with regard to FIGS. 45-48,the camera control may be used in remote control technology such as aremote surgery, remote control of a vehicle, an airplane, or aircraft,fighter, or remote control of construction, investigation or automaticassembly equipments.

METHOD AND SYSTEM FOR CONNTROLLING SPACE MAGNIFICATION FOR STEREOSCOPICIMAGES

[0283]FIG. 49 illustrates a 3D display system according to anotheraspect of the invention. The 3D display system is directed to adjustspace magnification for a stereoscopic image based on the spacemagnification adjusting data provided by a viewer.

[0284] The system comprises a camera site and a display site. Thedisplay site comprises an input device 4910, a set of display devices4920 and 4930, a transmitter 4950, and a pair of receivers 4940 and4960.

[0285] The input device 4910 provides a viewer's eye distance value (We)as shown in FIG. 43A and space magnification adjusting data to at leastone of the display devices 4920 and 4930. The space magnification meansthe size of space that a viewer perceives from the display images. Forexample, if the space magnification is “1,” a viewer perceives the samesize of the space in the display site as that of the real space that wasphotographed in the camera site. Also, if the space magnification is“10,” a viewer perceives ten times of the size of the space in thedisplay site larger than that of the real space that was imaged by thecamera . In addition, if the space magnification is “0.1,” a viewerperceives ten times the size of the space in the display site less thanthat of the real space that was imaged by the camera. The spacemagnification adjusting data represent data regarding the spacemagnification that a viewer wants to adjust. In one embodiment of theinvention, the space magnification adjusting data may comprise “0.1”times of space magnification, “1” times of space magnification, “10”times of space magnification, or “100” times of space magnification. Theadjustment of the space magnification is performed by an adjustment ofthe distance between the cameras 30 and 32, and will be described inmore detail later.

[0286] At least one of the display devices 4920 and 4930 displays thespace magnification adjusting data that are provided through the inputdevice 4910. The at least one of the display devices 4920 and 4930provides the space magnification adjusting data and eye distance value(W_(a)) to the transmitter 4950. The transmitter 4950 transmits themagnification adjusting data and the value W_(a) to the camera site. Inone embodiment of the invention, the space magnification adjusting dataand the value W_(a) may be provided directly from the input device 4910to the transmitter 4950 without passing through the display devices 4920and 4930.

[0287] The receiver 4970 receives the space magnification adjusting dataand W_(a) from the transmitter 4950, and provides the data to the cameracontrollers 4990. The camera controller 4990 controls the cameradistance based on the space magnification adjusting data and the valueW_(a). The camera controller 4990 comprises a servo controller 4985 anda horizontal motor 4975 as shown in FIG. 50. Referring to FIGS. 50-52,the operation of the camera controller 4990 will be explained.

[0288] The servo controller 4985 initializes camera distance (C₁), forexample, such that C₁ is the same as W_(a) (5100). The spacemagnification relates to the camera distance (C₁) and the eye distancevalue (W_(a)). When C₁ is the same as W_(a), the space magnification is“1,” which means that a viewer sees the same size of the object that isphotographed by the cameras 30 and 32. When C₁ is greater than W_(a),the space magnification is less than “1,” which means that a viewerperceives a smaller space than a space that is imaged by the cameras 30and 32. When C₁ is less than W_(a), the space magnification is greaterthan “1,” which means that a viewer perceives a larger sized object thanis imaged by the cameras 30 and 32.

[0289] The space magnification adjusting data (SM) are provided to theservo controller 4985 (5110). It is determined whether the adjustingdata is “1” (5120). If the adjusting data are “1,” no adjustment of thecamera distance is made (5160). If the adjusting data are not “1,” it isdetermined whether the adjusting data is greater than “1.” If theadjusting data are greater than “1,” the servo controller 4985 operatesthe motor 4975 so as to narrow C, until the requested spacemagnification is obtained (5150). Referring to FIG. 52, a table showingthe relationship between the space magnification and camera distance(C₁) is illustrated, where W_(a) is 80 mm. Thus, when C₁ is 80 mm, thespace magnification is “1.” In this situation, if the requested spacemagnification is “10,” the camera distance is adjusted to “8 mm” asshown in FIG. 52.

[0290] If the adjusting data are less than “1,” the servo controller4985 operates the motor 4975 so as to widen C₁ until the requested spacemagnification is obtained (5140). As exemplified in FIG. 52, if therequested space magnification is “0.1,” the camera distance is adjustedto “800 mm.”

[0291] Referring to FIG. 53, the operation of the entire system shown inFIG. 49 will be described. Stereoscopic images are displayed through thedisplay devices 4920 and 4930 (5010). Eye distance (W_(a)) and spacemagnification adjusting data (SM) are provided to the at least one ofthe display devices 4920 and 4930, or to the transmitter 4950 directlyfrom the input device 4910 (5020). The eye distance (W_(a)) and spacemagnification adjusting data (SM) are transmitted to the camera site(5030). The camera site receives the W_(a) and SM values and adjusts thecamera distance (C₁) based on the W_(a) and SM values (5040). Thestereoscopic cameras 30 and 32 image the object with adjusted spacemagnification (5050). The image is transmitted to the display sitethrough the transmitters 4980 and 5000 (5060). Each of the displaydevices 4920 and 4930 receives and displays the image (5070).

[0292] Regarding the embodiments described with regard to FIGS. 49-53,the camera control may be used in remote control technology such as aremote surgery, remote control of a vehicle, an airplane, or aircraft,fighter, or remote control of construction, investigation or automaticassembly equipments.

METHOD AND SYSTEM FOR ADJUSTING DISPLAY ANGLES OF STEREOSCOPIC IMAGEBASED ON A CAMERA LOCATION

[0293]FIG. 54 illustrates a 3D display system according to anotheraspect of the invention. The system is directed to adjust the locationof the display devices based on the relative location of thestereoscopic cameras with regard to an object 5400.

[0294] The system comprises a camera site and a display site. The camerasite comprises a set of stereoscopic cameras 30 and 32, a pair ofdirection detection devices 5410 and 5420, transmitters 5430 and 5440.In this embodiment of the invention, the cameras 30 and 32 may not beparallel to each other as shown in FIG. 54. The direction detectiondevices 5410 and 5420 detect directions of the stereoscopic cameras 30and 32 with respect to the object 5400 to be photographed, respectively.In one embodiment of the invention, the devices 5410 and 5420 detect thetilt angle with respect to an initial location where the two cameras areparallel to each other. In some situations, the cameras 30 and 32 may betilted, for example, 10 degrees in a counterclockwise direction as shownin FIG. 54, or in a clockwise direction from the initial location. Thedetection devices 5410 and 5420 detect the tilted angle of the cameras30 and 32, respectively. In one embodiment of the invention, each of thedirection detection devices 5410 and 5420 comprises a typical directionsensor.

[0295] Each of the transmitters 5430 and 5440 transmits the detecteddirection data of the cameras 30 and 32 to the display site. If it isdetected that only the camera 32 is tilted as shown in FIG. 57, thedetection device 5410 may not detect a tilting, and thus only thetransmitter 5440 may transmit the detected data to the display site. Thesame applies to a situation where only the camera 30 is tilted.

[0296] The display site comprises a pair of receivers 5450 and 5460, apair of display device controllers 5470 and 5500, and a set of displaydevices 5480 and 5490. Each of the receivers 5450 and 5460 receives thedetected tilting data of the cameras 30 and 32, and provides the data toeach of the display device controllers 5470 and 5500. The display devicecontrollers 5470 and 5500 determine display adjusting values based onthe received camera tilting data. The display adjusting values representmovement amounts to be adjusted for the display devices 5480 and 5490.In one embodiment of the invention, the display device controllers 5470and 5500 determine display adjusting values based on a table as shown inFIG. 55. In this embodiment of the invention, if the camera 32 is tilted10 degrees in a counter clockwise direction as shown in FIG. 54, thedisplay device controller 5500 tilts the corresponding display device5490 as much as 10 degrees in a clockwise direction as shown in FIG. 54.In this way, the camera location with respect to the object 5400 issubstantially the same as an eye lens location of the viewer with regardto the screen. As discussed above, the screen may comprise a V shapedmirror, a HMD screen, a projection screen, or a display screen 160 shownin FIG. 1B.

[0297] Referring to FIG. 56, the entire operation of the system shown inFIG. 54 will be explained. The set of stereoscopic cameras 30 and 32image an object (5510). Each of the direction detection devices 5410 and5420 detects a camera direction with respect to the object (5520). Thatis, for example, the degree of tilting of each camera 30 and 32 from,for example, a parallel state is detected. The photographed image data(PID) and direction detection data (DDD) are transmitted to the displaysite (5530). The PID and DDD are received in the display site, and theDDD are retrieved from the received data (5540, 5550). In one embodimentof the invention, the retrieving may be performed using a typical signalseparator.

[0298] At least one of the display device controllers 5470 and 5500determines the display device adjusting values based on the retrievedDDD (5560). The at least one of the display device controllers 5470 and5500 adjusts the display angle with respect to the viewer's eye lensesby moving a corresponding display device (5570). The display devices5480 and 5490 display the received stereoscopic images (5580).

[0299]FIG. 57 illustrates a 3D display system according to anotheraspect of the invention. The system is directed to adjust displayedimage based on the relative location of the stereoscopic cameras 30 and32 with regard to the object 5400.

[0300] The system shown in FIG. 57 is substantially the same as the oneof FIG. 54 except for the display devices 5710 and 5720. The displaydevices 5710 and 5720 adjust the location of the displayed images basedon the received camera direction detection data. Referring to FIG. 58,an exemplary block diagram of the display device 5720 is illustrated.Though not shown, the display device 5710 is substantially the same asthe display device 5720. The display device 5720 comprises amicrocomputer 5910, a memory 5920, a display driver 5930, and a displayscreen 5940. The memory 5920 stores a table (not shown) showing therelationship between the camera tilting angle and the adjust amount ofdisplayed images. The microcomputer 5910 determines display imageadjusting values based on the received camera direction data and thetable of the memory 5920. The display driver 5930 adjusts the displayangle of the display image based on the determined adjusting values, anddisplays the image in the display screen 5940.

[0301] Referring to FIGS. 59A and 59B, adjustment of the displayed imageis illustrated. In one embodiment of the invention, this may beperformed by enlarging or reducing the image portion of the left orright sides of the displayed image. For example, according to thetilting angle of the camera, the enlarging or reducing amount isdetermined. In this embodiment of the invention, enlargement orreduction may be performed by a known image reduction or magnificationsoftware. The image of FIG. 59A may correspond to the tilting of thedisplay device in a clockwise direction. Similarly, the image of FIG.59B may correspond to the tiling of the display device in a counterclockwise direction.

[0302] Referring to FIG. 60, the operation of FIG. 54 will be explained.As seen in FIG. 60, procedures 5810-5850 are the same as those shown inFIG. 55. Display image adjusting values are determined based on theretrieved camera direction detection data (DDD) (5860). The image to bedisplayed is adjusted as shown in FIG. 59 based on the determinedadjusting values (5870). The adjusted image is displayed (5880).

METHOD AND SYSTEM FOR TRANSMITTING OR STORING ON A PERSISTENT MEMORYSTEREOSCOPIC IMAGES AND PHOTOGRAPHING RATIOS

[0303]FIG. 61 illustrates a 3D display system according to anotheraspect of the invention. In this aspect of the invention, stereoscopicimages and photographing ratios are transmitted via a network such asthe Internet, or stored on a persistent memory, such as optical ormagnetic disks.

[0304] Referring to FIG. 61, the combined data 620 of stereoscopicimages 624 and at least one photographing ratio (A:B:C) 622 for theimages 624 are shown. The stereoscopic images 624 may comprisestereoscopic broadcasting images, stereoscopic advertisement images, orstereoscopic movie images, stereoscopic product images for Internetshopping, or any other kind of stereoscopic images. In one embodiment ofthe invention, the photographing ratio 622 may be fixed for the entireset of stereoscopic images 624. A method of combining of thestereoscopic images 624 and photographing ratio 622 has been describedabove in connection with FIG. 7.

[0305] In one embodiment, stereoscopic images 624 are produced from apair of stereoscopic cameras (not shown) and combined with thephotographing ratio 622. In one embodiment of the invention, thestereoscopic (broadcasting, advertisement, or movie, etc.) images 624and the photographing ratio 622 may be transmitted from an Internetserver, or a computing device of a broadcasting company. The Internetserver may be operated by an Internet broadcasting company, an Internetmovie company, an Internet advertising company or an Internet shoppingmall company. In another embodiment, the photographing ratio is notcombined, and rather, is transmitted separately from the stereoscopicimages. However, for convenience, the explanation below will be mainlydirected to the combined method.

[0306] The combined data 620 are transmitted to a computing device 627at a display site via a network 625. In one embodiment of the invention,the network 625 may comprise the Internet, a cable, a PSTN, or awireless network. Referring to FIG. 63, an exemplary data format of thecombined data 620 is illustrated. The left images and right images ofthe stereoscopic images 624 are embedded into the combined data 620 suchthat the images 624 are retrieved sequentially in a set of displaydevices 626 and 628. For example, left image I and right image 1, leftimage 2 and right image 2, are located in sequence in the data formatsuch that the images can be retrieved in that sequence. In oneembodiment, the computing device 627 receives the combined data 620 andretrieves the stereoscopic images 624 and photographing ratio 622 fromthe received data. In another embodiment, the images 624 andphotographing ratio 622 are separately received as they are not combinedin transmission.

[0307] The computing device 627 also provides the left and right imagesto the display device 626 and 628, respectively. In one embodiment ofthe invention, the data format may be constituted such that thecomputing device 627 can identify the left and right images of thestereoscopic images 624 when the device 627 retrieves the images 624such as predetermined order or data tagging. In one embodiment of theinvention, the computing device 627 may comprise any kind of computingdevices that can download the images 624 and ratio 622 either in acombined format or separately via the network 625. In one embodiment, apair of computing devices each retrieving and providing left and rightimages to the display devices 626 and 628, respectively may be providedin the display site.

[0308] The display devices 626 and 628 display the received stereoscopicimages such that the screen ratios (D1:E1:F1, D2:E2:F2) of each of thedisplay devices 626 and 628 are substantially the same as thephotographing ratio (A:B:C). In one embodiment of the invention, thescreen ratios (D1:E1:F1, D2:E2:F2) are the same(D1:E1:F1=D2:E2:F2=D:E:F). The display devices 626 and 628 may comprisethe elements of the display devices 86 and 88 disclosed in FIG. 8. Inone embodiment of the invention, each of the display devices 626 and 628may comprise CRT, LCD, HMD, PDP devices, or projection type displaydevices.

[0309] In another embodiment of the invention, as shown in FIG. 62, thecombined data which are stored in a recording medium 630 such as opticalor magnetic disks may be provided to the display devices 634 and 636 viaa medium retrieval device 632 at the display site. In one embodiment,the optical disks may comprise a compact disk (CD) or a digitalversatile disk (DVD). Also, the magnetic disk may comprise a hard disk.

[0310] The recording medium 630 is inserted into the medium retrievaldevice 632 that retrieves the stereoscopic images 624 and photographingratio 622. In one embodiment of the invention, the medium retrievaldevice 632 may comprise a CD ROM driver, a DVD ROM driver, or a harddisk driver (HDD), and a host computer for the drivers. The mediumretrieval device 632 may be embedded in a computing device (not shown).

[0311] The medium retrieval device 632 retrieves and provides thestereoscopic images 624 and photographing ratio 622 to the displaydevices 634 and 636, respectively. The exemplified data format shown inFIG. 63 may apply to the data stored in the recording medium 630. In oneembodiment of the invention, the photographing ratio 622 is the same forthe entire stereoscopic images. In this embodiment, the photographingratio 622 is provided once to each of the display devices 634 and 636,and the same photographing ratio is used throughout the stereoscopicimages.

[0312] In one embodiment of the invention, the data format recorded inthe medium 630 is constituted such that the medium retrieval device 632can identify the left and right images of the stereoscopic images 624.The operation of the display devices 634 and 636 is substantially thesame as that of the devices 626 and 628 as discussed with regard to FIG.61.

PORTABLE COMMUNICATION DEVICE COMPRISING A PAIR OF DIGITAL CAMERAS THATPRODUCE STEREOSCOPIC IMAGES AND A PAIR OF DISPLAY SCREENS

[0313]FIG. 64 illustrates an information communication system accordingto another aspect of the invention. The system comprises a pair ofportable communication devices 65 and 67. The device 65 comprises a pairof digital cameras 640, 642, a pair of display screens 644, 646, adistance input portion 648, an eye interval input portion 650, and aspace magnification input portion 652. The device 65 comprises areceiver and a transmitter, or a transceiver (all not shown).

[0314] The pair of digital cameras 640 and 642 produce stereoscopicimages of a scene or an object and photographing ratios thereof. In oneembodiment of the invention, each of the cameras 640 and 642 comprisessubstantially the same elements of the camera 20 shown in FIG. 7. Thedevice 65 transmits the produced stereoscopic images and photographingratios to the device 67. The pair of display screens 644 and 646 displaystereoscopic images received from the device 67.

[0315] The distance input portion 648 is provided with the distancevalues (similar to screen-viewer distances F1 and F2 in FIG. 8) betweena viewer's eyes and each of the screens 644 and 646. The eye intervalinput portion 650 receives the distance values (exemplified as W_(a) inFIG. 14A) between the center points of a viewer's eyes. The spacemagnification input portion 652 is provided with adjusting data forspace magnification, and provides the adjusting data to the device 65.In one embodiment of the invention, each of the distance input portion648, the eye interval input portion 645, and the space magnificationinput portion 652 comprises key pads that can input numerals 0-9. Inanother embodiment, all of the input portions are embodied as one inputdevice.

[0316] The device 67 comprises a pair of digital cameras 664, 666, apair of display screens 654, 656, a distance input portion 658, an eyeinterval input portion 660, and a space magnification input portion 662.The device 67 also comprises a receiver and a transmitter, or atransceiver (all not shown).

[0317] The pair of digital cameras 664 and 666 produce stereoscopicimages of a scene or an object and photographing ratios thereof. In oneembodiment of the invention, each of the cameras 664 and 666 comprisessubstantially the same elements of the camera 20 shown in FIG. 7. Thedevice 67 transmits the produced stereoscopic images and photographingratios to the device 65. The pair of display screens 654 and 656 displaystereoscopic images received from the device 65.

[0318] The distance input portion 658, the eye interval input portion660, and the space magnification input portion 662 are substantially thesame as those of the device 65.

[0319] The system shown in FIG. 64 may comprise at least one basestation (not shown) communicating with the devices 65 and 67. In oneembodiment of the invention, each of the devices 65 and 67 comprises acellular phone, an IMT (international mobile telecommunication)-2000device, and a personal digital assistant (PDA), a hand-held PC oranother type of portable telecommunication device.

[0320] In one embodiment of the invention, the space magnificationadjusting data and photographing ratios have a standard data format sothat the devices 65 and 67 can identify the data easily.

The devices Displaying Stereoscopic Images are Implemented such that thePhotographing Ratio is Substantially the Same as the Screen Ratio

[0321]FIG. 65 illustrates a pair of information communication devices 65and 67 according to one aspect of the invention. Each of the devices 65and 67 displays stereoscopic images received from the other device suchthat the photographing ratio of one device is substantially the same asthe screen ratio of the other device. The device 65 comprises a cameraportion 700, a display portion 720, and a data processor 740, e.g., amicrocomputer.

[0322] The camera portion 700 produces and transmits stereoscopic imagesand photographing ratios thereof to the device 67. As discussed above,the communication between the devices 65 and 67 may be performed via atleast one base station (not shown). The camera portion 700 comprises thepair of digital cameras 640, 642, and a transmitter 710. Each of thedigital cameras 640 and 642 produces stereoscopic images andphotographing ratios thereof, and combines the images and ratios(combined data 702 and 704). In one embodiment of the invention, thephotographing ratios provided in the combined data 702 and 704 are thesame. Each of the digital cameras 640 and 642 may comprise the elementsof the camera 20 shown in FIG. 7.

[0323] The production of the stereoscopic images and the calculation ofthe photographing ratios, and the combining of the images and ratioshave been explained in detail with regard to FIGS. 5-11. The transmitter710 transmits the combined data 702, 704 to the device 67. In anotherembodiment, the photographing ratios are not combined, and rather, aretransmitted separately from the stereoscopic images.

[0324] In one embodiment of the invention, the transmitter 710 maycomprise two transmitting portions that transmit the combined data 702and 704, respectively. The device 67 receives and displays thestereoscopic images transmitted from the device 65 such that thereceived photographing ratio is substantially the same as the screenratio of the device 67.

[0325] The display portion 720 receives combined data 714 and 716 ofstereoscopic images and photographing ratios thereof from the device 67,and displays the stereoscopic images such that the receivedphotographing ratio is substantially the same as the screen ratio of thedevice 65.

[0326] The display portion 720 comprises a pair of display devices 706,708, and a receiver 712. The receiver 712 receives the combined data 714and 716 that the device 67 transmitted, and provides the combined data714, 716 to the display devices 706, 708, respectively. In oneembodiment of the invention, the receiver 712 may comprise two receivingportions that receive the combined data 714 and 716, respectively. Inanother embodiment, the images and photographing ratios are separatelyreceived as they are not combined in transmission.

[0327] Each of the display devices 706 and 708 separates the providedimages and ratios from the receiver 712. The devices 706 and 708 alsodisplay the stereoscopic images such that the photographing ratios aresubstantially the same as the screen ratios of the display devices 706and 708, respectively. Each of the display devices 706 and 708 maycomprise substantially the same elements of the display device 86 or 88shown in FIG. 8. In one embodiment, the display devices 706 and 708 areconnected to the distance input portion 648 shown in FIG. 64 so that thescreen-viewer distance for the devices 706 and 708 can be provided tothe device 65. In one embodiment of the invention, the screen ratios forthe devices 706 and 708 are substantially the same. The detailedoperation of the display devices 706 and 708 has been explained inconnection with FIGS. 8-11.

[0328] The microcomputer 740 controls the operation of the cameraportion 700 and display portion 720, and data communication with thedevice 67. In one embodiment of the invention, the microcomputer 740 isprogrammed to control the camera portion 700 such that the digitalcameras 640 and 642 produce stereoscopic images and photographing ratiosthereof, and that the transmitter 710 transmits the images and ratios tothe device 67 when the communication link is established between thedevices 65 and 67. In another embodiment of the invention, themicrocomputer 740 is programmed to control the power of the cameraportion 700 and the display portion 720 independently. In thisembodiment, even when the cameras 640 and 642 are turned off, thedisplay devices 706 and 708 may display the stereoscopic images receivedfrom the device 67. Also, when the display devices 706 and 708 areturned off, the cameras 640 and 642 may produce stereoscopic images andphotographing ratios thereof, and transmit the images and ratios to thedevice 67. In this embodiment, the device 65 may comprise an elementthat performs a voice signal communication with the device 67.

[0329] The device 65 may include a volatile memory such as a RAM and/ora non-volatile memory such as a flash memory or a programmable ROM thatstore data for the communication. The device 65 may comprise a powersupply portion such as a battery.

[0330] In another embodiment of the invention, the device 65 may includea transceiver that incorporates the transmitter 710 and receiver 712. Inthis situation, the transmitter 710 and receiver 712 may be omitted.

[0331] Though not specifically shown, the device 67 may be configured tocomprise substantially the same elements and perform substantially thesame functions as those of the device 65 shown in FIG. 65. Thus, thedetailed explanation of embodiments thereof will be omitted.

The Devices Controlling the Display Location of the Stereoscopic Images

[0332]FIG. 66A illustrates an information communication device 65according to another aspect of the invention. In this aspect of theinvention, the information communication device 65 controls the displaylocation of the stereoscopic images based on the distance (W_(a))between the center points of a viewer's eyes.

[0333] In one embodiment of the invention, the device 65 moves thestereoscopic images displayed in the display screens 644 and 646 suchthat the distance (W_(d)) between the center points of the displayedstereoscopic images is substantially the same as the W_(a) distance. Thedevice 65 comprises an eye interval input portion 650, a data processor722, e.g., a microcomputer, a pair of display drivers 724, 726, and apair of display screens 644, 646. The eye interval input portion 650 andthe pair of display screens 644 and 646 are substantially the same asthose of FIG. 64.

[0334] The microcomputer 722 controls the display drivers 724 and 726based on the received W_(a) distance such that the W_(d) distance issubstantially the same as the W_(a) distance. Specifically, the displaydrivers 724 and 726 moves the stereoscopic images displayed in thedisplay screens 644 and 646 until W_(d) is substantially the same asW_(a). The detailed explanation with regard to the movement of thestereoscopic images has been provided in connection with FIGS. 15-17.

[0335] In another embodiment of the invention, as shown in FIG. 66B, thedevice 65 moves the display screens 644 and 646 such that the distance(W_(d)) between the center points of the stereoscopic images issubstantially the same as the W_(a) distance. In this embodiment, thedevice 67 comprises the eye interval input portion 650, a microcomputer732, a pair of servo mechanisms 734, 736, and the pair of displayscreens 644, 646.

[0336] The microcomputer 732 controls the servo mechanisms 734 and 736based on the received W_(a) distance such that the W_(d) distance issubstantially the same as the W_(a) distance. Specifically, the servomechanisms 734 and 736 move the display screens 644 and 646 until W_(d)is substantially the same as W_(a). The detailed explanation with regardto the movement of the display screens has been provided with regard toFIGS. 18-20.

[0337] Though not specifically shown, the device 67 may comprisesubstantially the same elements and performs substantially the samefunctions as those of the device 65 shown in FIGS. 66A and 66B. Thus,the detailed explanation of embodiments thereof will be omitted.

[0338] The devices adjusting space magnification of stereoscopic imagesFIG. 67 illustrates an information communication device 65 according toanother aspect of the invention. In this aspect of the invention, theinformation communication device 65 adjusts space magnification based onadjusting data for space magnification. The device 65 comprises a cameraportion 760, a display portion 780, and a microcomputer 750.

[0339] The camera portion 760 comprises a pair of digital cameras 640,642, a camera controller 742, and a transceiver 744. The transceiver 744receives adjusting data for space magnification from the device 67, andprovides the adjusting data (C) to the camera controller 742. Spacemagnification embodiments have been explained in detail with respect toFIGS. 49-53. The adjusting data for space magnification are exemplifiedin FIG. 52.

[0340] The camera controller 742 controls the distance (interval)between the digital cameras 640 and 642 based on the provided adjustingdata (C). In one embodiment of the invention, the camera controller 742comprises a motor that adjusts the camera distance, and a servocontroller that controls the motor (both not shown). The operation ofthe camera controller 742 is substantially the same as that of thecontroller 4990 described in connection with FIGS. 50-52. The digitalcameras 640 and 642 produce stereoscopic images in adjusted interval,and transmit the stereoscopic images to the device 67 through thetransceiver 744. The device 67 receives and displays the adjustedstereoscopic images. In this way, the device 67 can adjust the spacemagnification for a scene imaged by the cameras 640, 642 of the device65. In one embodiment of the invention, each of the devices 65 and 67may display in at least one of the display screens thereof current spacemagnification, such as “1”, “0.5” or “10,” etc., so that a viewer canknow the current space magnification. In another embodiment of theinvention, the devices 65 and 67 may provide a user with an audio signalrepresenting the current space magnification.

[0341] In another embodiment, space magnification adjusting data (A) maybe provided to the camera controller 742, for example, through the spacemagnification input portion 652 shown in FIG. 64. This embodiment may beuseful in a situation where a user of the device 65 wants to providestereoscopic images in adjusted space magnification to a user of thedevice 67. In one embodiment, the operation of the camera controller 742is substantially the same as in a situation where the adjusting data (C)is received from the device 67.

[0342] The display portion 780 comprises a pair of display screens 644,646, and a transceiver 746. Space magnification (SM) adjusting data (B)are provided to the transceiver 746 from a user of the device 65. The SMadjusting data (B) are used to adjust the interval between the cameras664 and 666 of the device 67 (FIG. 64). The SM adjusting data (B) mayalso be provided to at least one of the display screens 644 and 646 sothat the SM adjusting data (B) are displayed in the at least one of thedisplay screens 644 and 646. This is to inform a user of the device 65of current space magnification. The transceiver 746 transmits the SMadjusting data (B) to the device 67.

[0343] The device 67 receives the SM adjusting data (B) and adjusts theinterval between the cameras 664 and 666 of the device 67 based on theadjusting data (B). Also, the device 67 transmits stereoscopic imagesproduced in adjusted space magnification to the device 65. Thetransceiver 746 receives left and right images from the device 67 andprovides the images to the display screens 644 and 646, respectively.The display screens 644 and 646 display the stereoscopic images. In oneembodiment, each of the devices 65 and 67 of FIG. 67 may furthercomprise the functions of the devices 65 and 67 described in connectionwith FIGS. 65 and 66.

[0344] The microcomputer 750 controls the operation of the cameraportion 760 and display portion 780, and data communication with thedevice 67. In one embodiment of the invention, the microcomputer 750 isprogrammed to control the camera portion 760 and display portion 780such that after the communication link between the devices 65 and 67 isestablished, the SM adjusting data (B, C) are transmitted or receivedfrom or to each other. In another embodiment of the invention, themicrocomputer 750 is programmed to control the camera portion 760 suchthat the camera controller 742 adjusts the interval between the digitalcameras 640 and 642 based on the SM adjusting data (A) even when thecommunication link between the devices 65 and 67 is not established.

[0345] The device 65 may include a volatile memory such as a RAM and/ora non-volatile memory such as a flash memory or a programmable ROM thatstore data for the communication. The device 65 may comprise an elementthat performs a voice signal transmission.

[0346] Though not specifically shown, embodiments of the device 67comprise substantially the same elements and perform the same functionsas those of the device 65 shown in FIG. 67. Thus, a detailed explanationof these embodiments will be omitted.

The Device Comprising Separate Display Screens

[0347] In another embodiment of the invention, the communication device65 comprises a goggle shaped display device 649 as shown in FIG. 68. Thegoggle shaped display device comprises a set of display screens 645 and647. In one embodiment of the invention, the display device 649 may beconnected to the device 65 through a communication jack 643. In anotherembodiment of the invention, the display device 649 may have a wirelessconnection to the device 65.

[0348] The device 67 may be applied to the embodiments described withregard to FIGS. 65-67. In one embodiment of the invention, each of thedevices 65 and 67 may comprise a head mount display (HMD) device thatincludes a set of display screens.

OTHER ASPECTS OF THE INVENTION

[0349]FIG. 69 illustrates a 3D display system according to anotheraspect of the invention. In this aspect of the invention, stereoscopicimages are produced from three-dimensional structural data. Thethree-dimensional structural data may comprise 3D game data or 3Danimation data.

[0350] As one example, the three-dimensional structural data comprisepixel values (e.g., RGB pixel values) ranging from, for example, (0000,0000, 0000) to (9999, 9999, 9999) in the locations from (000, 000, 000)to (999, 999, 999) in a 3D coordinate system (x, y, z). In thissituation, Table 1 exemplifies data #1—data #N of the 3D structuraldata. TABLE 1 Data #1 Data #2 . . . Data #N in a location in a locationin a location (001, 004, 002) (001, 004, 004) (025, 400, 087) (0001,0003, 1348) (0010, 0033, 1234) . . . (0001, 3003, 1274)

[0351] In one embodiment of the invention, as shown in FIG. 69A,stereoscopic images are produced from three-dimensional structural data752 in a remote server. The three-dimensional structural data 752 areprojected into a pair of two dimensional planes using known projectionportions 754 and 756, which are also frequently referred to as imaginarycameras or view points in stereoscopic image display technology. Theprojection portions may comprise a know software that performs theprojection function. These projected images are stereoscopic images,each comprising a pair of two-dimensional plane images that aretransmitted to a display site. In the display site, the stereoscopicimages are displayed in a pair of display devices.

[0352] In another embodiment of the invention, as shown in FIG. 69A,stereoscopic images are produced from three-dimensional structural datain a display site. In this embodiment, the three-dimensional structuraldata may be transmitted or downloaded from a remote server to thedisplay site. The projection portions 772 and 774 are located in acomputing device 770. In one embodiment of the invention, the projectionportions 772 and 774 may comprise a software module and be downloadedwith the structural data from the remote server to the computing device770 of the display site. The projected images, i.e., producedstereoscopic images are displayed through a pair of display devices 776and 778. In another embodiment of the invention, the 3D structural dataare stored on a recording medium such as optical disks or magnetic disksand inserted and retrieved in the computing device 770 as discussed withregard to FIG. 62. In this situation, a software module for theprojection portions 772 and 774 may be included in the medium.

[0353] A method of producing stereoscopic images from thethree-dimensional structural data is, for example, disclosed in U.S.Pat. No. 6,005,607, issued Dec. 21, 1999, which is incorporated byreference herein.

[0354] This aspect of the invention may be applied to all of the aspectsof the invention described above. In some embodiments, however, somemodification may be made. As one example, the photographing ratios ofthe imaginary cameras (projection portions, view points) may becalculated by calculating horizontal and vertical lengths of aphotographed object or scene and the distance between the cameras andthe object (scene), using the location of the cameras and object in theprojected coordinate system.

[0355] As another example, the control of the motions of the imaginarycameras may be performed by a computer software that identifies thelocation of the imaginary cameras and controls the movement of thecameras.

[0356] As another example, the control of the space magnification may beperformed by adjusting the interval between the imaginary cameras usingthe identified location of the imaginary cameras in the projectedcoordinate system.

[0357]FIG. 70 illustrates a 3D display system according to anotheraspect of the invention. This aspect of the invention is directed todisplay stereoscopic images such that the resolution of each displaydevice is substantially the same as that of each stereoscopic camera. Inthis aspect of the invention, the locations of the pixels that arephotographed in each camera with regard to a camera frame (e.g.,640×480) are substantially the same as those of the pixels that aredisplayed in each display device with regard to a display screen (e.g.,1280×960). Referring to FIG. 70, the resolution of the display device isdouble that of the camera. Thus, one pixel of the left top comerphotographed in the camera is converted to four pixels of the displayscreen in the same location as shown in FIG. 70. Similarly, one pixel ofthe right bottom comer photographed in the camera is converted to fourpixels of the display screen in the same location as shown in FIG. 70.This aspect of the invention may be applied to all of the 3D displaysystems described in this application.

[0358] The above systems have been described showing a communicationlocation connecting the display to a remote camera site. However, thesevarious inventions can be practiced without a receiver/a transmitter andnetwork so that functions are performed at a single site. Some of theabove systems also have been described based on a viewer's eye lensmotion or location. However, the systems can be practiced based on aviewer's eye pupils or corneas.

[0359] While the above description has pointed out novel features of theinvention as applied to various embodiments, the skilled person willunderstand that various omissions, substitutions, and changes in theform and details of the device or process illustrated may be madewithout departing from the scope of the invention. Therefore, the scopeof the invention is defined by the appended claims rather than by theforegoing description. All variations coming within the meaning andrange of equivalency of the claims are embraced within their scope.

What is claimed is:
 1. A method of displaying stereoscopic images,comprising: displaying at least one stereoscopic image on a set ofdisplay device, the stereoscopic image comprising a pair oftwo-dimensional plane images; providing adjustment data for spatialmagnification, the spatial magnification relating to a size of a spacethat is imaged; transmitting the magnification adjustment data to a setof remotely located stereoscopic cameras; and adjusting the distancebetween the stereoscopic cameras based on the magnification adjustmentdata.
 2. The method of claim 1, further comprising: imaging a sceneusing the adjusted cameras; transmitting the image to the set of displaydevice; and receiving and displaying the image on the set of displaydevice.
 3. The method of claim 1, further comprising: providing adistance (W_(a)) between the center points of a viewer's eyes;initializing the camera distance to be the same as the W_(a) value;determining whether the magnification adjustment data is greater than 1;and narrowing the camera distance if the magnification adjustment datais greater than
 1. 4. The method of claim 3, further comprising wideningthe camera distance if the magnification adjustment data is less than 1.5. The method of claim 1, further comprising displaying the adjustedspatial magnification on the set of display device.
 6. The method ofclaim 1, further comprising providing a voice signal representing theadjusted spatial magnification.
 7. The method of claim 1, furthercomprising producing the at least one stereoscopic image fromthree-dimensional structural data.
 8. The method of claim 1, wherein theset of display device comprises a unitary display device adapted tosequentially the two-dimensional plane images.
 9. The method of claim 1,wherein the set of display device comprises a pair of display devicesconfigured to display simultaneously the two-dimensional plane images,respectively.
 10. A system for displaying stereoscopic images,comprising: a set of display device configured to display at least onestereoscopic image, the stereoscopic image comprising a pair oftwo-dimensional plane images; an input device configured to provideadjustment data for spatial magnification, the spatial magnificationrelating to a size of a space that is imaged; a transmitter configuredto transmit the magnification adjustment data; a receiver configured toreceive the magnification adjustment data; a set of stereoscopiccameras; and a camera controller configured to adjust the distancebetween the stereoscopic cameras based on the magnification adjustmentdata.
 11. The system of claim 10, wherein the camera controllercomprises: a servo controller configured to determine an adjustmentamount based on the spatial magnification adjustment data; and ahorizontal motor configured to adjust the camera distance based on thedetermined adjustment amount.
 12. The system of claim 10, wherein theset of display device is selected from one of the following: a headmount display, a set of projection display device, a set of LCD device,a set of CRT device, and a set of plasma display panel device.
 13. Thesystem of claim 10, wherein the set of the display device is configuredto display the adjusted spatial magnification.
 14. The system of claim10, wherein the set of display device comprises a unitary display deviceadapted to sequentially the two-dimensional plane images.
 15. The systemof claim 10, wherein the set of display device comprises a pair ofdisplay devices configured to display simultaneously the two-dimensionalplane images, respectively.
 16. A camera system for communicating datawith a set of display device, comprising: a set of stereoscopic camerasconfigured to produce at least one stereoscopic image; a receiverconfigured to receive adjustment data for spatial magnification from theset of display device, the space magnification relating to a size of thespace that is imaged by the set of stereoscopic cameras; and a cameracontroller configured to adjust the distance between the set ofstereoscopic cameras based on the spatial magnification adjustment data.17. The camera system of claim 16, further comprising a transmitterconfigured to transmit the adjusted stereoscopic image to the set ofdisplay device.
 18. The camera system of claim 16, wherein the receiveris configured to receive a distance (W_(a)) between the center points ofa viewer's eyes and the camera controller is configured to adjust thecamera distance based on the magnification adjustment data and the W_(a)value.
 19. A method of controlling a set of stereoscopic cameras,comprising: providing a set of stereoscopic cameras; generating at leastone stereoscopic image, the stereoscopic image comprising a pair oftwo-dimensional plane images generated by the set of stereoscopiccameras, respectively; receiving adjustment data for spatialmagnification, the spatial magnification relating to a size of the spacethat is imaged by the set of stereoscopic cameras; and adjusting thedistance between the stereoscopic cameras based on the spatialmagnification adjustment data.
 20. The method of claim 19, furthercomprising: initializing the camera distance such that the spatialmagnification is 1; determining whether the magnification adjustmentdata is greater than 1; and narrowing the camera distance if themagnification adjustment data is greater than
 1. 21. The method of claim20, further comprising widening the camera distance if the magnificationadjustment data is less than
 1. 22. A method of displaying stereoscopicimages, comprising: providing a pair of projection portions spaced at apredetermined distance apart from each other and configured to projectthree-dimensional structural data of a scene into a pair oftwo-dimensional planes; producing at least one stereoscopic image fromthe three-dimensional structural data by the pair of projection portionsspaced at a predetermined distance apart from each other, thestereoscopic image comprising a pair of two-dimensional plane images;displaying the pair of two-dimensional plane images in a pair of displaydevices, respectively; providing adjustment data for space magnificationto the display devices, the space magnification relating to a size ofthe scene; and adjusting the distance between the projection portionsbased on the magnification adjustment data.
 23. The method of claim 22,wherein the spatial magnification is related to a size of the scene thatis perceived by a viewer.
 24. A system for displaying stereoscopicimages, comprising: a pair of projection portions spaced at apredetermined distance apart from each other and configured to produceat least one stereoscopic image from three-dimensional structural dataof a scene, the stereoscopic image comprising a pair of two-dimensionalplane images; a pair of display devices configured to display the pairof two-dimensional plane images, respectively; an input deviceconfigured to provide adjustment data for spatial magnification, thespatial magnification relating to a size of the scene; and a computingdevice configured to adjust the distance between the projection portionsbased on the magnification adjustment data.
 25. A system for controllinga set of stereoscopic cameras, comprising: means for providing a set ofstereoscopic cameras; means for generating at least one stereoscopicimage, the stereoscopic image comprising a pair of two-dimensional planeimages generated by the set of stereoscopic cameras, respectively; meansfor receiving adjustment data for spatial magnification, the spatialmagnification relating to a size of the space that is imaged by the setof stereoscopic cameras; and means for adjusting the distance betweenthe stereoscopic cameras based on the spatial magnification adjustmentdata.